Behaviour Pattern Analysis System, Mobile Terminal, Behaviour Pattern Analysis Method, and Program

ABSTRACT

Provided is a mobile terminal including a movement sensor that detects a movement of a user and outputs movement information, acquires information on a building existing at a current location or information on buildings existing in a vicinity of the current location, analyses the movement information output from the movement sensor, and detects a first behaviour pattern corresponding to the movement information from multiple first behaviour patterns obtained by classifying behaviours performed by the user over a relatively short period of time, and analyses the information on a building or buildings and the first behaviour pattern, and detects a second behaviour pattern corresponding to the information on a building or buildings and the first behaviour pattern from multiple second behaviour patterns obtained by classifying behaviours performed by the user over a relatively long period of time.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a behaviour pattern analysis system, amobile terminal, a behaviour pattern analysis method, and a program.

2. Description of the Related Art

A technology for installing a motion sensor on a mobile terminal such asa mobile phone and for enabling automatic detection and recording of auser's behaviour history is gaining attention. For example,JP-A-2008-3655 discloses a technology for detecting a walking movement,a running movement, a movement of turning left or right and a stillstate by using a motion sensor such as an acceleration sensor and a gyrosensor. This patent document describes a method of calculating a walkingspeed, a walking power and a rotation angle around a gravity axis fromoutput data of the motion sensor, and detecting the walking movement,the running movement, the movement of turning left or right and thestill state by using the calculation result. Furthermore, this patentdocument describes a method of detecting a user's behaviour pattern bystatistical processing which has, as inputs, the pattern of the movementor the state such as the type of the movement or the state, the durationof the movement or the state and the number of times of the movement.

SUMMARY OF THE INVENTION

Using the method described above, a behaviour pattern such as “slowwalking” and “busy movement” can be obtained as time series data.However, the behaviour pattern obtained by this method mainly expressesa relatively short-term movement or state of a user. Accordingly, it isdifficult to estimate, from a behaviour pattern history, a specificcontent of a behaviour such as “today, went shopping at a departmentstore” or “yesterday, had a meal at a restaurant in a hotel.” Thebehaviour pattern obtained by using the method described in the patentdocument described above is an accumulation of behaviours performed in arelatively short period of time. Also, each of the behavioursconstituting the behaviour pattern is not purposefully performed by auser. On the other hand, a specific content of a behaviour is, in manycases, that of a behaviour that is purposefully performed by a user andthat is performed for entertainment over a relatively long period oftime. Accordingly, it is difficult to know such specific content of abehaviour from an accumulation of behaviours performed in a short periodof time.

In light of the foregoing, it is desirable to provide a behaviourpattern analysis system, a mobile terminal, a behaviour pattern analysismethod, and a program, which are new and improved, and which are capableof detecting, from a relatively short-term behaviour pattern obtained byusing a motion sensor, a relatively long-term highly-entertainingbehaviour pattern.

According to an embodiment of the present invention, there is provided abehaviour pattern analysis system which includes a mobile terminalincluding a movement sensor that detects a movement of a user andoutputs movement information, a current location information acquisitionunit that acquires information on a current location, a buildinginformation acquisition unit that acquires information on a buildingexisting at a location indicated by the information acquired by thecurrent location information acquisition unit or information onbuildings existing at the current location and in a vicinity of thecurrent location, a first behaviour pattern detection unit that analysesthe movement information output from the movement sensor, and detects afirst behaviour pattern corresponding to the movement information frommultiple first behaviour patterns obtained by classifying behavioursperformed by the user over a relatively short period of time, and atransmission unit that transmits, to a server, the information on abuilding or buildings acquired by the building information acquisitionunit and the first behaviour pattern detected by the first behaviourpattern detection unit, and a server including a reception unit thatreceives, from the mobile terminal, the information on a building orbuildings and the first behaviour pattern, and a second behaviourpattern detection unit that analyses the information on a building orbuildings and the first behaviour pattern received by the receptionunit, and detects a second behaviour pattern corresponding to theinformation on a building or buildings and the first behaviour patternfrom multiple second behaviour patterns obtained by classifyingbehaviours performed by the user over a relatively long period of time.

The second behaviour pattern detection unit may create, by using aspecific machine learning algorithm, a detection model for detecting thesecond behaviour pattern from the information on a building or buildingsand the first behaviour pattern, and may detect, by using the createddetection model, the second behaviour pattern corresponding to theinformation on a building or buildings and the first behaviour patternreceived by the reception unit.

The mobile terminal may further include a time information acquisitionunit that acquires information on a time of a time point of acquisitionof the information on a current location by the current locationinformation acquisition unit.

The transmission unit may transmit, to the server, the information on abuilding or buildings acquired by the building information acquisitionunit, the first behaviour pattern detected by the first behaviourpattern detection unit and the information on a time acquired by thetime information acquisition unit. The server may hold, for eachcombination of the first behaviour pattern and the information on atime, a score map assigning a score to each combination of theinformation on a building or buildings and the second behaviour pattern.In a case the score map is selected based on the first behaviour patterndetected by the first behaviour pattern detection unit and theinformation on a time acquired by the time information acquisition unit,a combination of scores corresponding to the information on a buildingexisting at the current location acquired by the building informationacquisition unit is extracted from the selected score map and a highestscore in the extracted combination of scores is a specific value orless, the second behaviour pattern detection unit may create, by using aspecific machine learning algorithm, a detection model for detecting thesecond behaviour pattern from the information on a building or buildingsand the first behaviour pattern, and may detect, by using the createddetection model, the second behaviour pattern corresponding to theinformation on a building or buildings and the first behaviour patternreceived by the reception unit.

The mobile terminal may further include a storage unit in which scheduleinformation recording, in a time-series manner, a behaviour of a usercapable of being expressed by a combination of the second behaviourpatterns is stored, a matching determination unit that reads theschedule information stored in the storage unit, and determines whethera present behaviour, a past behaviour and a future behaviour of the userrecorded in the schedule information and the second behaviour patterndetected by the second behaviour pattern detection unit match or not,and a display unit that displays, according to a result of determinationby the matching determination unit, whether an actual behaviour matchesa schedule recorded in the schedule information, is behind the schedule,or is ahead of the schedule.

The server may further include a behaviour prediction unit thatpredicts, by using a history of the second behaviour patterns detectedby the second behaviour pattern detection unit, a second behaviourpattern to be performed by the user next.

In a case of determining that a behaviour of the user matching thesecond behaviour pattern is not recorded in the schedule information,the matching determination unit may acquire, from the server, the secondbehaviour pattern predicted by the behaviour prediction unit and mayextract, from the schedule information, a behaviour of the user matchingthe acquired second behaviour pattern. The display unit may displayinformation relating to the behaviour of the user extracted by thematching determination unit.

The server may include a location information accumulation unit thatreceives, by the reception unit, the information on a current locationacquired by the current location information acquisition unit and thefirst behaviour pattern detected by the first behaviour patterndetection unit, and stores the information on a current location and ahistory of the first behaviour patterns in the storage unit, and aclustering unit that clusters places where the user stays for a longtime, by using the information on a current location and the history ofthe first behaviour patterns accumulated in the storage unit by thelocation information accumulation unit, and calculates a stayingprobability of staying at each of the places and a movement probabilityof moving between the places. The behaviour prediction unit may predictthe second behaviour pattern to be performed by the user next, based onthe staying probability and the movement probability calculated by theclustering unit.

The behaviour pattern analysis system may include multiple mobileterminals. The server may further include a notification informationstorage unit that stores, in association with each other, notificationinformation to be notified at a specific time and a specific secondbehaviour pattern, and an information notification unit that, at thespecific time, refers to the second behaviour pattern detected by thesecond behaviour pattern detection unit based on the information on abuilding or buildings and the first behaviour pattern received by thereception unit from each of the mobile terminals, and notifies a mobileterminal corresponding to a second behaviour pattern same as thespecific second behaviour pattern of the notification information.

The information notification unit may count the number of mobileterminals corresponding to a second behaviour pattern same as thespecific second behaviour pattern, and in a case the number of themobile terminals is a specific number or more, may notify all of themultiple mobile terminals of the notification information.

According to another embodiment of the present invention, there isprovided a mobile terminal which includes a movement sensor that detectsa movement of a user and outputs movement information, a currentlocation information acquisition unit that acquires information on acurrent location, a building information acquisition unit that acquiresinformation on a building existing at a location indicated by theinformation acquired by the current location information acquisitionunit or information on buildings existing at the current location and ina vicinity of the current location, a first behaviour pattern detectionunit that analyses the movement information output from the movementsensor, and detects a first behaviour pattern corresponding to themovement information from multiple first behaviour patterns obtained byclassifying behaviours performed by the user over a relatively shortperiod of time, and a second behaviour pattern detection unit thatanalyses the information on a building or buildings acquired by thebuilding information acquisition unit and the first behaviour patterndetected by the first behaviour pattern detection unit, and detects asecond behaviour pattern corresponding to the information on a buildingor buildings and the first behaviour pattern from multiple secondbehaviour patterns obtained by classifying behaviours performed by theuser over a relatively long period of time.

The mobile terminal may further include a time information acquisitionunit that acquires information on a time of a time point of acquisitionof the information of a current location by the current locationinformation acquisition unit. A score map assigning a score to eachcombination of the information on a building or buildings and the secondbehaviour pattern may be provided for each combination of the firstbehaviour pattern and the information on a time. The second behaviourpattern detection unit may select the score map based on the firstbehaviour pattern detected by the first behaviour pattern detection unitand the information on a time acquired by the time informationacquisition unit, may extract, from the selected score map, acombination of scores corresponding to the information on a buildingexisting at the current location acquired by the building informationacquisition unit, and may detect the second behaviour patterncorresponding to a highest score in the extracted combination of scores.

The mobile terminal may further include a time information acquisitionunit that acquires information on a time of a time point of acquisitionof the information of a current location by the current locationinformation acquisition unit. A score map assigning a score to eachcombination of the information on a building or buildings and the secondbehaviour pattern may be provided for each combination of the firstbehaviour pattern and the information on a time. The buildinginformation acquisition unit may acquire, as the information onbuildings existing at the current location and in a vicinity of thecurrent location, category types of the buildings and the number ofbuildings for each category type. The second behaviour pattern detectionunit may select the score map based on the first behaviour patterndetected by the first behaviour pattern detection unit and theinformation on a time acquired by the time information acquisition unit,may extract, from the selected score map, combinations of scorescorresponding to respective category types acquired by the buildinginformation acquisition unit, may normalise, by respective highestscores, each score included in the combinations of scores correspondingto respective category types, may performs weighting on the normalisedcombinations of scores corresponding to respective category typesaccording to the number of buildings for each category type acquired bythe building information acquisition unit, and may add, for each secondbehaviour pattern, the weighted scores corresponding to the respectivecategory types, and detects the second behaviour pattern for which aresult of addition is greatest.

The mobile terminal may further include a display unit on which adisplay object for starting an application associated with the secondbehaviour pattern is displayed, and a display control unit that makesthe display unit preferentially display, according to the secondbehaviour pattern detected by the second behaviour pattern detectionunit, the display object associated with the second behaviour pattern.

According to another embodiment of the present invention, there isprovided a behaviour pattern analysis server which includes a receptionunit that receives, from a mobile terminal including a movement sensorthat detects a movement of a user and outputs movement information and acurrent location information acquisition unit that acquires informationon a current location, the movement information and the information on acurrent location, a building information acquisition unit that acquiresinformation on a building existing at a location indicated by theinformation on a current location received by the reception unit orinformation on buildings existing at the current location and in avicinity of the current location, a first behaviour pattern detectionunit that analyses the movement information received by the receptionunit, and detects a first behaviour pattern corresponding to themovement information from multiple first behaviour patterns obtained byclassifying behaviours performed by the user over a relatively shortperiod of time, and a second behaviour pattern detection unit thatanalyses the information on a building or buildings acquired by thebuilding information acquisition unit and the first behaviour patterndetected by the first behaviour pattern detection unit, and detects asecond behaviour pattern corresponding to the information on a buildingor buildings and the first behaviour pattern from multiple secondbehaviour patterns obtained by classifying behaviours performed by theuser over a relatively long period of time.

According to another embodiment of the present invention, there isprovided a behaviour pattern analysis method which includes the steps ofacquiring movement information indicating a result of detection by amovement sensor for detecting a movement of a user, acquiringinformation on a current location, acquiring information on a buildingexisting at a location indicated by the information on a currentlocation acquired in the step of acquiring information on a currentlocation or information on buildings existing at the current locationand in a vicinity of the current location, analysing the movementinformation acquired in the step of acquiring movement information, anddetecting a first behaviour pattern corresponding to the movementinformation from multiple first behaviour patterns obtained byclassifying behaviours performed by the user over a relatively shortperiod of time, and analysing the information on a building or buildingsacquired in the step of acquiring information on a building or buildingsand the first behaviour pattern detected in the step of analysing themovement information and detecting a first behaviour pattern, anddetecting a second behaviour pattern corresponding to the information ona building or buildings and the first behaviour pattern from multiplesecond behaviour patterns obtained by classifying behaviours performedby the user over a relatively long period of time.

According to another embodiment of the present invention, there isprovided a program for causing a computer to realise a movementinformation acquisition function of acquiring movement informationindicating a result of detection by a movement sensor for detecting amovement of a user, a current location information acquisition functionof acquiring information on a current location, a building informationacquisition function of acquiring information on a building existing atthe current location indicated by the information acquired by thecurrent location information acquisition function or information onbuildings existing at the current location and in a vicinity of thecurrent location, a first behaviour pattern detection function ofanalysing the movement information acquired by the movement informationacquisition function, and detecting a first behaviour patterncorresponding to the movement information from multiple first behaviourpatterns obtained by classifying behaviours performed by the user over arelatively short period of time, and a second behaviour patterndetection function of analysing the information on a building orbuildings acquired by the building information acquisition function andthe first behaviour pattern detected by the first behaviour patterndetection function, and detecting a second behaviour patterncorresponding to the information on a building or buildings and thefirst behaviour pattern from multiple second behaviour patterns obtainedby classifying behaviours performed by the user over a relatively longperiod of time.

According to another embodiment of the present invention, there isprovided a recording medium in which the program is recorded, therecording medium being able to be read by a computer.

According to the embodiments of the present invention described above, arelatively long-term highly-entertaining behaviour pattern can bedetected from a relatively short-term behaviour pattern obtained byusing a motion sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an explanatory diagram showing an example of the systemconfiguration of a behaviour/situation analysis system according to thefirst embodiment of the present invention;

FIG. 2 is an explanatory diagram showing an example of the systemconfiguration of the behaviour/situation analysis system according tothe embodiment;

FIG. 3 is an explanatory diagram showing an example of the systemconfiguration of the behaviour/situation analysis system according tothe embodiment;

FIG. 4 is an explanatory diagram showing an example of the systemconfiguration of the behaviour/situation analysis system according tothe embodiment;

FIG. 5 is an explanatory diagram showing an example of the systemconfiguration of the behaviour/situation analysis system according tothe embodiment;

FIG. 6 is an explanatory diagram showing an example of the systemconfiguration of the behaviour/situation analysis system according tothe embodiment;

FIG. 7 is an explanatory diagram for explaining a function of amovement/state recognition unit according to the embodiment;

FIG. 8 is an explanatory diagram for explaining a function of themovement/state recognition unit according to the embodiment;

FIG. 9 is an explanatory diagram for explaining a function of themovement/state recognition unit according to the embodiment;

FIG. 10 is an explanatory diagram for explaining a function of themovement/state recognition unit according to the embodiment;

FIG. 11 is an explanatory diagram for explaining a function of themovement/state recognition unit according to the embodiment;

FIG. 12 is an explanatory diagram for explaining a function of themovement/state recognition unit according to the embodiment;

FIG. 13 is an explanatory diagram for explaining a function of themovement/state recognition unit according to the embodiment;

FIG. 14 is an explanatory diagram for explaining a function of themovement/state recognition unit according to the embodiment;

FIG. 15 is an explanatory diagram for explaining a function of themovement/state recognition unit according to the embodiment;

FIG. 16 is an explanatory diagram for explaining a function of ageo-categorisation unit according to the embodiment;

FIG. 17 is an explanatory diagram for explaining a function of thegeo-categorisation unit according to the embodiment;

FIG. 18 is an explanatory diagram for explaining a function of thegeo-categorisation unit according to the embodiment;

FIG. 19 is an explanatory diagram for explaining a function of thegeo-categorisation unit according to the embodiment;

FIG. 20 is an explanatory diagram for explaining a function of abehaviour/situation recognition unit according to the embodiment;

FIG. 21 is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 22 is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 23 is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 24 is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 25 is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 26 is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 27 is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 28 is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 29 is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 30A is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 30B is an explanatory diagram for explaining a function of thebehaviour/situation recognition unit according to the embodiment;

FIG. 31 is an explanatory diagram showing an overview of a function of abehaviour/situation analysis system according to the second embodimentof the present invention;

FIG. 32 is an explanatory diagram showing an overview of a function ofthe behaviour/situation analysis system according to the embodiment;

FIG. 33 is an explanatory diagram showing an example of the systemconfiguration of the behaviour/situation analysis system according tothe embodiment;

FIG. 34 is an explanatory diagram for explaining a function of abehaviour prediction unit according to the embodiment;

FIG. 35A is an explanatory diagram showing a flow of processingperformed prior to processing by a behaviour verification unit accordingto the embodiment;

FIG. 35B is an explanatory diagram showing a flow of processingperformed by the behaviour verification unit according to theembodiment;

FIG. 35C is an explanatory diagram showing a flow of processingperformed by the behaviour verification unit according to theembodiment;

FIG. 36 is an explanatory diagram showing an overview of a function of abehaviour/situation analysis system according to the third embodiment ofthe present invention;

FIG. 37 is an explanatory diagram showing an overview of a function ofthe behaviour/situation analysis system according to the embodiment;

FIG. 38 is an explanatory diagram showing an example of the systemconfiguration of the behaviour/situation analysis system according tothe embodiment;

FIG. 39 is an explanatory diagram for explaining a function of a ToDomanagement unit according to the embodiment;

FIG. 40 is an explanatory diagram for explaining a function of the ToDomanagement unit according to the embodiment;

FIG. 41 is an explanatory diagram for explaining a function of the ToDomanagement unit according to the embodiment;

FIG. 42A is an explanatory diagram for explaining a function of the ToDomanagement unit according to the embodiment;

FIG. 42B is an explanatory diagram for explaining a function of the ToDomanagement unit according to the embodiment;

FIG. 42C is an explanatory diagram for explaining a function of the ToDomanagement unit according to the embodiment;

FIG. 42D is an explanatory diagram for explaining a function of the ToDomanagement unit according to the embodiment;

FIG. 43 is an explanatory diagram for explaining a function of the ToDomanagement unit according to the embodiment;

FIG. 44 is an explanatory diagram for explaining a function of the ToDomanagement unit according to the embodiment;

FIG. 45 is an explanatory diagram showing an overview of a function of abehaviour/situation analysis system according to the fourth embodimentof the present invention;

FIG. 46 is an explanatory diagram showing an example of the systemconfiguration of the behaviour/situation analysis system according tothe embodiment; and

FIG. 47 is an explanatory diagram showing an example of the hardwareconfiguration of an information processing apparatus capable ofrealising functions of a server and a client configuring thebehaviour/situation analysis systems according to the first to fourthembodiments of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

<Flow of Description>

The flow of description of embodiments of the present inventiondescribed below will be briefly mentioned here. First, a configurationof a behaviour/situation analysis system according to the firstembodiment of the present invention will be described with reference toFIGS. 1 to 30B. Then, a configuration of a behaviour/situation analysissystem according to the second embodiment of the present invention willbe described with reference to FIGS. 31 to 35C. Then, a configuration ofa behaviour/situation analysis system according to the third embodimentof the present invention will be described with reference to FIGS. 36 to44. Then, a configuration of a behaviour/situation analysis systemaccording to the fourth embodiment of the present invention will bedescribed with reference to FIGS. 45 and 46. Then, an example of thehardware configuration of an information processing apparatus capable ofrealising functions of a server and a client configuring thebehaviour/situation analysis systems according to the first to fourthembodiments of the present invention will be described with reference toFIG. 47.

(Description Items)

1: First Embodiment

1-1: System Configuration

1-2: Function of Movement/State Recognition Unit 108

1-3: Function of Geo-Categorisation Unit 110

1-4: Function of Behaviour/Situation Recognition Unit 112

2: Second Embodiment

2-1: Overview of System

2-2: Overall Configuration of System

2-3: Function of Behaviour Prediction Unit 208

2-4: Function of Behaviour Verification Unit 206

3: Third Embodiment

3-1: Overview of System

3-2: Overall Configuration of System

3-3: Function of ToDo Management Unit 304

4: Fourth Embodiment

4-1: Overview of System

4-2: Overall Configuration of System

5: Hardware Configuration 1: First Embodiment

The first embodiment of the present invention will be described. Thepresent embodiment relates to a technology of detecting a behaviour anda situation of a user by using information on the user's movement andstate detected by using a motion sensor and location informationdetected by a location sensor. Additionally, as the motion sensor, athree-axis acceleration sensor (including an acceleration sensor, agravity detection sensor, a fall detection sensor, and the like), athree-axis gyro sensor (including an angular velocity sensor, ahand-blur compensation sensor, a geomagnetic sensor, and the like), andthe like, are used, for example. Also, as the location sensor, a GPS(Global Positioning System) is used, for example. However, since thelatitude and longitude of the current location can be detected from anRFID) (Radio Frequency Identification), a Wi-Fi access point,information on a wireless base station, and the like, and such detectionmeans can also be used as the location sensor.

<1-1: System Configuration>

First, a system configuration of a behaviour/situation analysis system10 according to the present embodiment will be described with referenceto FIG. 1. FIG. 1 is an explanatory diagram showing an example of anoverall system configuration of the behaviour/situation analysis system10 according to the present embodiment. Additionally, in thisspecification, expressions “movement, state” and “behaviour, situation”will be used differently with respect to the following meanings.

(1) The expression “movement, state” means a behaviour performed by auser which is relatively short-term, lasting several seconds to severalminutes, and indicates an action such as “walking,” “running,” “jumping”or “still,” for example. Furthermore, these actions will be collectivelyexpressed as “movement/state pattern” or “LC (Low-Context) behaviour.”On the other hand, (2) the expression “behaviour, situation” is a dailybehaviour performed by a user over a longer period of time than with“movement, state,” and indicates an action such as “meal,” “shopping” or“work,” for example. Furthermore, these actions will be collectivelyexpressed as “behaviour/situation pattern” or “HC (High-Context)behaviour.”

Now, as shown in FIG. 1, the behaviour/situation analysis system 10 ismainly configured from a motion sensor 102, a location sensor 104, atime/calendar information acquisition unit 106, a movement/staterecognition unit 108, a geo-categorisation unit 110, and abehaviour/situation recognition unit 112. Furthermore, an application APand a service SV that use a behaviour/situation pattern that is detectedby the behaviour/state recognition unit 112 are prepared for thebehaviour/situation analysis system 10. Furthermore, a result of usageof the behaviour/situation pattern by the application AP and profileinformation of a user may be input to the behaviour/situationrecognition unit 112.

When a user performs a behaviour, first, a change in acceleration,rotation around a gravity axis and the like (hereinafter, sensor data)are detected by the motion sensor 102. The sensor data detected by themotion sensor 102 is input to the movement/state recognition unit 108.Furthermore, location information indicating the whereabouts(hereinafter, current location) of a user is acquired by the locationsensor 104. The current location is expressed by latitude and longitude,for example. Furthermore, the location information on the currentlocation acquired by the location sensor 104 is input to thegeo-categorisation unit 110.

The movement/state recognition unit 108 is means for detecting amovement/state pattern by using the sensor data. Accordingly, when thesensor data is input from the motion sensor 102, the movement/staterecognition unit 108 detects a behaviour/state pattern based on theinput sensor data. A movement/state pattern that can be detected by themovement/state recognition unit 108 is “walking,” “running,” “still,”“jumping,” “train (aboard/not aboard)” and “elevator (aboard/notaboard/ascending/descending),” for example. Additionally, amovement/state pattern detection method of the movement/staterecognition unit 108 will be described later in detail (with referenceto FIGS. 7 to 15). However, the movement/state pattern detection methodis not limited to the example described later, and a method that usesmachine learning can also be adopted. Moreover, the movement/statepattern detected by the movement/detection recognition unit 108 is inputto the behaviour/situation recognition unit 112.

The geo-categorisation unit 110 is means for acquiring map informationMP, and detecting an attribute of the current location indicated in theinput location information by using the acquires map information MP Notethat the geo-categorisation unit 110 uses a geo category code as meansfor expressing the attribute of the current location. The geo categorycode is a classification code for classifying types of pieces ofinformation relating to a place (see FIG. 17). Also, this geo categoryis set according to the type of a building, the shape of a landscape,geographical characteristics or regional characteristics, for example.Accordingly, by specifying the geo category code of the currentlocation, the environment a user is in can be recognised to a certaindegree.

Accordingly, the geo categorisation unit 110 refers to the acquired mapinformation MP, specifies a building or the like existing at the currentlocation based on the location information input from the locationsensor 104, and selects a geo category code corresponding to thebuilding or the like. The geo category code selected by thegeo-categorisation unit 110 is input to the behaviour/situationrecognition unit 112. Additionally, when using the environmentsurrounding the current location for the detection of thebehaviour/situation pattern, the geo-categorisation unit 110 selects geocategory codes corresponding to multiple buildings or the like existingin the vicinity of the current location, and inputs the selected geocategory codes or pieces of information based thereon (see FIG. 18) tothe behaviour/situation recognition unit 112.

As described above, the movement/state pattern and the geo category codeare input to the behaviour/situation recognition unit 112 respectivelyfrom the movement/state recognition unit 108 and the geo-categorisationunit 110. Furthermore, the sensor data is also input to thebehaviour/situation recognition unit 112 via the movement/staterecognition unit 108. Furthermore, the location information on thecurrent location is also input to the behaviour/situation recognitionunit 112 via the geo-categorisation unit 110. Furthermore, time/calendarinformation is input to the behaviour/situation recognition unit 112 viathe time/calendar information acquisition unit 106. This time/calendarinformation is information indicating the time the sensor data isacquired by the motion sensor 102 and the time the location informationis acquired by the location sensor 104. For example, time/calendarinformation includes information on the time the sensor data or thelocation information is acquired, information on the day of the week,information on a holiday, information on the date, or the like.

Accordingly, the behaviour/situation recognition unit 112 detects thebehaviour/situation pattern based on the movement/state pattern, the geocategory code (or information based thereon) and the time/calendarinformation that have been input. The behaviour/situation pattern isdetected by using (1) determination processing based on a rule(hereinafter, rule-based determination) or (2) determination processingbased on a learning model (hereinafter, learning model determination).

(1) Rule-Based Determination

The rule-based determination is a method of assigning a score to eachcombination of a geo category code and a behaviour/situation pattern anddetermining an appropriate behaviour/situation pattern corresponding toinput data based on the score. A rule for assigning a score is asexpressed by a score map SM (see FIG. 21). The score map SM is preparedfor each piece of the time/calendar information, such as the date, thetime or the day of the week. For example, a score map SM correspondingto Monday of the first week of March is prepared. Furthermore, the scoremap SM is prepared for each movement/state pattern, such as walking,running or train. For example, a score map SM for walking is prepared.Therefore, the score map is prepared for each combination of thetime/calendar information and the movement/state pattern. Accordingly,multiple score maps SM are prepared.

Accordingly, the behaviour/situation recognition unit 112 selects, fromthe multiple score maps SM that have been prepared, a score map SMmatching the time/calendar information and the movement/state pattern.Then, the behaviour/situation recognition unit 112 extracts a scorecorresponding to the geo category code from the selected score map SM.Additionally, the order of processing can be changed as appropriate withrespect to the process of selecting a score map SM based on thetime/calendar information and the movement/state pattern and the processof extracting a score based on the geo category code. By theseprocesses, the behaviour/situation recognition unit 112 can extract thescore of each behaviour/situation pattern existing in the score map SMwhile taking into consideration the situation of the current location atthe time point of acquisition of the sensor data.

Furthermore, the behaviour/situation recognition unit 112 specifies thehighest score from the extracted scores, and extracts thebehaviour/situation pattern corresponding to the highest score. Thismethod of detecting the behaviour/situation pattern in this manner isthe rule-based determination. Additionally, the score in the score mapSM shows a probability of a user being presumed to act according to thebehaviour/situation pattern corresponding to the score. That is, thescore map SM shows a score distribution of the behaviour/situationpatterns according to which a user is presumed to act under thesituation of the current location indicated by the geo category code.

For example, the probability of a user in a department store at threeo'clock on Sunday doing “shopping” is presumed to be high. However, theprobability of a user in the same department store at around seveno'clock in the evening “having a meal” is also presumed to be high. Asdescribed, the score map SM (to be more precise, a score map SM group)shows the score distribution of a user's behaviour/situation patterns atcertain times at certain locations. For example, the score map SM may beinput in advance by the user or a third party, may be obtained by usingmachine learning, or may be built by using other statistical method.Also, the score map SM may be optimised by personal profile informationPR or a behaviour/situation feedback FB obtained from the user. Theprofile information PR includes age, sex, occupation, information onhome, and information on workplace, for example. Furthermore, thebehaviour/situation feedback FB includes information indicating whethera behaviour/situation pattern that is output is correct or not.

(2) Learning Model Determination

The learning model determination is a method of creating a determinationmodel for determination of the behaviour/situation pattern using amachine learning algorithm and of determining the behaviour/situationpattern corresponding to input data by using the created determinationmodel (see FIG. 25). Additionally, as the machine learning algorithm,the k-means method, the Nearest Neighbor method, the SVM method, the HMMmethod or the Boosting method can be used, for example. The SVM is anabbreviation for Support Vector Machine. Also, the HMM is anabbreviation for Hidden Markov Model. In addition to these methods,there is also a method of creating the determination model by using analgorithm-building method based on a genetic search described inJP-A-2009-48266.

Additionally, as a feature quantity vector, the time/calendarinformation, the movement/state pattern, the geo category code (orinformation based thereon), the sensor data or the location informationon the current location is used, for example. However, in the case ofusing the algorithm-building method based on the genetic search, agenetic search algorithm is used at a stage of selecting, in the processof learning, a feature quantity vector. The behaviour/situationrecognition unit 112 first inputs to the machine learning algorithm afeature quantity vector for which the correct behaviour/situationpattern is known, as learning data, and creates a determination modelfor determining a reliability of each behaviour/situation pattern or theoptimal behaviour/situation pattern.

Then, the behaviour/situation recognition unit 112 inputs input data tothe created determination model, and determines a behaviour/situationpattern that is presumed to match the input data. Note that, in a case afeedback of true or false for a result of determination performed byusing the created determination model is obtained, the determinationmodel is rebuilt by using the feedback. Then, the behaviour/situationrecognition unit 112 determines the behaviour/situation pattern that ispresumed to match the input data, by using the determination model thathas been rebuilt. This method of detecting a behaviour/situation patternthat matches input data in this manner is the learning modeldetermination. Additionally, the amount of computation is larger for thelearning model determination than for the rule-based determination.

The behaviour/situation recognition unit 112 detects, by the methodsdescribed above, the behaviour/situation pattern that matches the inputdata input from the time/calendar information acquisition unit 106, themovement/state recognition unit 108 and the geo-categorisation unit 110.The behaviour/situation pattern detected by the behaviour/situationrecognition unit 112 is used for providing a recommended service SVcorresponding to the behaviour/situation pattern or is used by theapplication AP that performs processing according to thebehaviour/situation pattern. Concrete examples of the configuration ofthe recommended service SV and the application AP will be described ingreater detail in relation to the second to fourth embodiments describedlater.

Heretofore, an overall system configuration of the behaviour/situationanalysis system 10 according to the present embodiment has beendescribed with reference to FIG. 1. Next, a server/client configurationof the behaviour/situation system 10 will be described.

(Server/Client Configuration)

Each of the functions of the behaviour/situation analysis system 10shown in FIG. 1 is actually realised by a server or a client device (amobile terminal or the like). However, how to decide between a functionto be assigned to the server and a function to be assigned to the clientdevice is a matter that should be changed as appropriate according tothe arithmetic processing capability or the like of the server or theclient device. Here, an example of a server/client configuration will beintroduced (see FIGS. 2 to 6).

(System Configuration Example (1))

First, the system configuration example illustrated in FIG. 2 will beintroduced. In the example of FIG. 2, the motion sensor 102, thelocation sensor 104, the time/calendar information acquisition unit 106,the movement/state recognition unit 108 and the geo-categorisation unit110 are provided in the client device. Furthermore, the application APis installed in the client. On the other hand, the behaviour/situationrecognition unit 112 is provided in the server. Furthermore, the scoremap SM is held by the server.

The most important factor at the time of determining the server/clientconfiguration in the behaviour/situation analysis system 10 is theamount of computation of the behaviour/situation recognition unit 112.Particularly, realisation of the function of the behaviour/situationrecognition unit 112 by using the learning model determination is notpossible with the current computation capability of the client devicesuch as a mobile phone or portable game machine. Therefore, as shown inthe example of FIG. 2, in the case of using the learning modeldetermination, it is preferable that the behaviour/situation recognitionunit 112 is provided in the server and that the behaviour/situationpattern is acquired from the server and be used. Furthermore, in thecase the amount of data of the score map SM is large, the storagecapacity of the client device might be taken up, and thus the score mapSM is desirably held by the server.

Moreover, the motion sensor 102, the location sensor 104 and thetime/calendar information acquisition unit 106 are means for acquiringraw data that directly reflects the behaviour of a user. Furthermore,the application AP is means for providing, to a user, a function andinformation generated based on a detected behaviour/situation pattern.Accordingly, as shown in the example of FIG. 2, the motion sensor 102,the location sensor 104, the time/calendar information acquisition unit106 and the application AP have to be provided in the client server.

Furthermore, the map information MP and geo category information GC maybe held internally by the client device or may be acquired from theoutside. Particularly, the amount of data of the map information MP willbe extremely large in a case it is highly accurate. Therefore, a designshould be appropriately modified according to the storage capacity ofthe client device with regard to whether the map information MP is to beinternally held or is to be acquired from outside. Furthermore, in acase the latest map information MP is desired to be used at all times,it is preferable that map information MP existing outside the clientdevice can be used.

(System Configuration Example (2))

Next, the system configuration example illustrated in FIG. 3 will beintroduced. In the example of FIG. 3, the motion sensor 102, thelocation sensor 104, the time/calendar information acquisition unit 106,the movement/state recognition unit 108, the geo-categorisation unit110, the behaviour/situation recognition unit 112 and the application APare provided in the client device. Furthermore, the score map SM is heldby the client device.

As described above, when using the learning model determination, theamount of computation of the behaviour/situation recognition unit 112becomes extremely large, and realisation becomes difficult with thecomputation capability of an existing client device. However, in thecase of using the rule-based determination, the amount of computation ofthe behaviour/situation recognition unit 112 is relatively small, andthus the behaviour/situation recognition unit 112 can be provided in theclient device. Furthermore, in a case the computation capability of theclient device improves in the future or in a case a high-end PC(Personal Computer), a high-end game machine or the like is used as theclient device, it becomes possible to perform the learning modeldetermination by the client device. In this case, the server onlyprovides the recommended service SV according to the behaviour/situationpattern.

According to such configuration, communication between the client deviceand the server can be made less frequent, and the application AP thatperforms processing according to the behaviour/situation pattern can becomfortably used even in a poor communication environment. Furthermore,by exchanging feedbacks relating to the behaviour/situation patternsbetween the client devices, the behaviour/situation pattern of anotheruser living in a similar environment, such as a friend, a family memberor a co-worker, can be used as learning data. Accordingly, adetermination model for a behaviour/situation pattern or a score map SMmatching the living environment of a user is created, and the accuracyof the behaviour/situation pattern is improved.

(System Configuration Example (3))

Next, the system configuration example illustrated in FIG. 4 will beintroduced. In the example of FIG. 4, the motion sensor 102, thelocation sensor 104, the time/calendar information acquisition unit 106and the application AP are provided in the client device. Furthermore,the movement/state recognition unit 108, the geo-categorisation unit 110and the behaviour/situation recognition unit 112 are provided in theserver. Also, the score map SM is held by the server.

As described above, the amount of computation of the behaviour/situationrecognition unit 112 is relatively large. Particularly, in the case ofusing the learning model determination, it is difficult to realise thefunction of the behaviour/situation recognition unit 112 with thecomputation capability of an existing client device. On the other hand,the amount of computation of the movement/state recognition unit 108 andthe amount of computation of the geo-categorisation unit 110 arerelatively small. However, in a case of reducing the size of the clientdevice, a processor with small amount of heat generation and with smalldie size is sometimes installed at the expense of computationcapability. In such case, the movement/state recognition unit 108 andthe geo-categorisation unit 110 are preferably provided in the server asshown in the example of FIG. 4.

According to such configuration, the client device can allocatecomputation resources to the application AP or other functions.Furthermore, since the geo-categorisation unit 110 is not provided inthe client device, operations of holding or acquiring the mapinformation MP and the geo category information GC become unnecessary.Accordingly, a storage area that is temporarily or perpetually occupiedby the map information MP or the geo category information GC can bereleased.

(System Configuration Example (4))

Next, the system configuration example illustrated in FIG. 5 will beintroduced. In the example of FIG. 5, the motion sensor 102, thelocation sensor 104, the time/calendar information acquisition unit 106,the movement/state recognition unit 108, the geo-categorisation unit110, the behaviour/situation recognition unit 112 and the application APare provided in the client device. Furthermore, the score map SM orinformation on the determination model is held by the server.

As described above, the amount of computation of the behaviour/situationrecognition unit 112 is relatively large. Particularly, in the case ofusing the learning model determination, it is difficult to realise thefunction of the behaviour/situation recognition unit 112 with thecomputation capability of an existing client device. Accordingly, asshown in the example of FIG. 5, a method can be conceived of calculatingthe determination model in advance by the server and providing theclient device with the determination model. In this case, thebehaviour/situation recognition unit 112 provided in the client devicedetects the behaviour/situation pattern by using the determination modelprovided from the server as it is, or the behaviour/situationrecognition unit 112 modifies the determination model based on afeedback from a user and uses the modified determination model.According to such configuration, even a client device with relativelylow computation capability is enabled to realise the function of thebehaviour/situation recognition unit 112.

Also in a case of using the rule-based determination, the amount of dataof the score map SM sometimes takes up the storage area of the clientdevice. Accordingly, it is sometimes preferable that the score map SM isheld by the server. Furthermore, a process of optimising the score mapSM is also a process that calls for a relatively large amount ofcomputation. Accordingly, by optimising the score map SM by the serverand by using the optimised score map SM by the behaviour/situationrecognition unit 112 of the client device, highly accurate detection ofthe behaviour/situation pattern can be realised by the client device.

(System Configuration Example (5))

Next, the system configuration example illustrated in FIG. 6 will beintroduced. In the example of FIG. 6, the motion sensor 102, thelocation sensor 104, the time/calendar information acquisition unit 106,the movement/state recognition unit 108, the geo-categorisation unit110, the behaviour/situation recognition unit 112 and the application APare provided in the client device. Furthermore, the score map SM is heldby the client device. The system configuration illustrated in FIG. 6 ismostly the same with the system configuration example (2) describedabove. However, although the map information MP and the geo categoryinformation GC were held by the client device according to the systemconfiguration example (2), the map information MP and the geo categoryinformation GC are acquired from outside according to the systemconfiguration example (5) illustrated in FIG. 6. According to suchconfiguration, a usable storage area can be expanded. Also, since itbecomes possible to use the latest map information MP and geo categoryinformation GC, the behaviour/situation pattern can be detected withfurther improved accuracy.

Heretofore, examples of the server/client configuration of thebehaviour/situation analysis system 10 have been introduced.Additionally, the system configurations illustrated in FIGS. 2 to 6 canbe similarly applied to the server/client configurations of the secondto fourth embodiments described later. In the following, the function ofeach structural element configuring the behaviour/situation analysissystem 10 will be described in greater detail.

<1-2: Function of Movement/State Recognition Unit 108>

First, a function of the movement/state recognition unit 108 will bedescried in detail with reference to FIGS. 7 to 15. FIGS. 7 to 15 areexplanatory diagrams for explaining a function and an operation of themovement/state recognition unit 108.

(Structure of Input-Output Data)

Reference will be made to FIG. 7. As described above, sensor data isinput from the motion sensor 102 to the movement/state recognition unit108. For example, waveform data of acceleration (hereinafter,acceleration data) is input as the sensor data, as shown in FIG. 7.Additionally, although it may not be clear from the example of FIG. 7,the acceleration data in x direction (x-acc), the acceleration data in ydirection (y-acc) and the acceleration data in z direction (z-acc) areinput to the movement/state recognition unit 108. Additionally, x, y andz are directions mutually orthogonal to each other. Furthermore, in acase a gyro sensor is installed, three-dimensional gyro data (x-gyro,y-gyro, z-gyro) is input as the sensor data. Additionally, since thesensitivity of the sensor changes due to the temperature, atmosphericpressure or the like, these pieces of sensor data are preferablycalibrated.

When the sensor data is input, the movement/state recognition unit 108detects a movement/state pattern based on the input sensor data. Themovement/state pattern that can be detected by the movement/staterecognition unit 108 may be, for example, “walking,” “running,” “still,”pausing,” “jumping,” “posture changing,” “turning,” “train (aboard),”“elevator (ascending/descending),” “car (aboard)” or “bicycle (riding)”(see FIG. 8).

For example, an algorithm for detecting a walking state will beconsidered. Normally, the frequency of the acceleration data detected atthe time of a person walking is about 2 Hz (two steps per second).Accordingly, the movement/state recognition unit 108 analyses thefrequency of acceleration data, and detects a portion at which thefrequency is about 2 Hz. The portion detected by the process correspondsto the movement/state “walking.” The movement/state recognition unit 108can also detect, from the acceleration data, the time of occurrence ofthe “walking” movement/state pattern, the duration thereof, or the like.Furthermore, the movement/state recognition unit 108 can detect theintensity of “walking” from the amplitude of the acceleration data.

As described, a feature quantity of each movement/state pattern(hereinafter, movement/state feature quantity) can be detected based ondata such as frequency or intensity obtained by analysing the sensordata. Additionally, only the acceleration data is used in the example ofthe “walking” movement/state, but gyro data is also used depending onthe type of the movement/state pattern. When a change over time in themovement/state feature quantity is obtained, the movement/staterecognition unit 108 sequentially determines the movement/state patternsfrom the movement/state feature quantity, and outputs a movement/statepattern that changes time-serially (see FIG. 7). However, it should benoted that the movement/state pattern obtained here is only a collectionof actions of a user performed in a short period of time, and does notindicate a specific daily behaviour of the user.

The movement/state pattern obtained by the movement/state recognitionunit 108 in this manner is input to the behaviour/situation recognitionunit 112. Now, a more detailed detection algorithm will be describedbelow in relation to a part of the movement/state patterns shown in FIG.8.

(Pause/Stillness Recognition Method)

First, a method of recognising whether a user is pausing or stayingstill will be described with reference to FIG. 9. FIG. 9 is anexplanatory diagram showing a method of recognising whether a user ispausing or staying still.

First, when a user performs a behaviour, sensor data is input to themovement/state recognition unit 108. Here, three-axis directionacceleration data (x-acc, y-acc, z-acc) is input. When the sensor datais input, the movement/state recognition unit 108 records the sensordata in FIFO manner (S1000). When a specific amount of data is recorded,the movement/state recognition unit 108 calculates respective variancesof x-acc, y-acc and z-acc (S1002). Then, the movement/state recognitionunit 108 extracts a largest variance for stillness determination (det)which is the largest variance among the variances (S1004).

When the largest variance for stillness determination is extracted, themovement/state recognition unit 108 determines whether or not theextracted largest variance for stillness determination is smaller than astillness recognition value D1 indicating a still state (S1006). In acase the largest variance for stillness determination is not smallerthan D1, the movement/state recognition unit 108 determines that theuser is not staying still. In a case such determination is made, it isassumed that the user is performing some kind of behaviour. Accordingly,the movement/state recognition unit 108 inputs information indicatingthat the user is not staying still to the behaviour/situationrecognition unit 112 (S1014).

On the other hand, in a case the largest variance for stillnessdetermination is smaller than D1, the movement/state recognition unit108 determines whether or not the state where the largest variance issmaller than D1 continues for a period longer than a stillnessrecognition time T1 (S1008). The stillness recognition time T1 hereindicates the minimum time during which the user is assumed to bestaying still. In a case the state where the largest variance is smallerthan D1 continues for a period longer than T1, the movement/staterecognition unit 108 determines that the user is staying still, andinputs information indicating stillness to the behaviour/situationrecognition unit 112 (S1010). In a case the state where the largestvariance is smaller than D1 does not continue for a period longer thanT1, the movement/state recognition unit 108 determines that the user ispausing, and inputs information indicating pause to thebehaviour/situation recognition unit 112 (S1012).

As described, by performing determination processing according to theexample of FIG. 9, a still state, a pause state and a non-still statecan be distinguished.

(Walking/Running Recognition Method)

Next, a method of recognising whether a user is walking or running willbe described with reference to FIG. 10. FIG. 10 is an explanatorydiagram showing a method of recognising whether a user is walking orrunning.

First, when a user performs a behaviour, sensor data is input to themovement/state recognition unit 108. Here, three-axis directionacceleration data (x-acc, y-acc, z-acc) is input. When the sensor datais input, the movement/state recognition unit 108 removes, from x-acc,y-acc and z-acc, a frequency outside a frequency range at which the useris recognised to be walking or running, by using a band-pass filter(BPF) (S1100). Then, the movement/state recognition unit 108 recordsx-acc, y-acc and z-acc that passed through the BPF in FIFO manner(S1102).

Then, the movement/state recognition unit 108 reads specific amounts ofthe x-acc, y-acc and z-acc that are recorded after passing through theBPF, and calculates an autocorrelation function (SACF: summaryautocorrelation function) for the read out data. The time-series of SACFpeaks corresponds to a periodic movement of a user caused at the time ofwalking or running. However, the SACF includes a harmonic of a frequencycorresponding to walking or running. Accordingly, the movement/staterecognition unit 108 calculates an enhanced autocorrelation function(ESACF: enhanced summary autocorrelation function) based on thecalculated autocorrelation function (SACF) (S1106). Then, themovement/state recognition unit 108 calculates an autocorrelation peakbased on the ESACF (S1108), and obtains a frequency for walking/runningdetermination (freq).

Furthermore, the movement/state recognition unit 108 records, in FIFOmanner, x-acc, y-acc and z-acc that are not yet made to pass through theBPF in step S1100 (S1110). Then, the movement/state recognition unit 108reads specific amounts of the x-acc, y-acc and z-acc, and calculates therespective variances (S1112). Then, the movement/state recognition unit108 extracts the largest variance from the calculated variances, andoutputs the same as a largest variance for walking/running determination(var) (S1114).

Then, the movement/state recognition unit 108 multiplies the frequencyfor walking/running determination (freq) and the largest variance forwalking/running determination (var) (S1116). The number of steps perunit time is expressed by freq. Also, the magnitude of a movement isexpressed by var. Furthermore, whether a user is walking or running canbe determined based on the number of steps and the magnitude of themovement. Therefore, whether a user is walking or not can be determinedby determining whether a product of freq and var is within a specificrange or not. First, to increase the accuracy of the determination, themovement/state recognition unit 108 calculates data for walking/runningdetermination det by removing, from the product of freq and var, afrequency range at which walking or running is easily erroneouslyrecognised (S1118).

Then, the movement/state recognition unit 108 determines whether or notthe data for walking/running determination is larger than a minimumwalking recognition value D2 which is the lower limit for walking to berecognised but smaller than a largest walking recognition value D3 whichis the upper limit for walking to be recognised (S1120). In a case it islarger than D2 but smaller than D3, the movement/state recognition unit108 determines that the user is walking, and inputs informationindicating walking to the behaviour/situation recognition unit 112(S1122). On the other hand, in a case it is not D2<det<D3, themovement/state recognition unit 108 proceeds to the process of stepS1124, and determines whether or not the data for walking/runningdetermination det is larger than D3 (S1124).

In a case the data for walking/running determination is larger than D3,the movement/state recognition unit 108 determines that the user isrunning, and inputs information indicating running to thebehaviour/situation recognition unit 112 (S1126). On the other hand, ina case the data for walking/running determination is below D2, themovement/state recognition unit 108 determines that the user is netherwalking nor running, and inputs information that the movement/statepattern is not of walking or running to the behaviour/situationrecognition unit 112 (S1128). Additionally, by integrating the freq,information on the number of steps walked during a period of timecorresponding to an integral interval is obtained. Thus, themovement/state recognition unit 108 calculates the information on thenumber of steps, and inputs the information to the behaviour/situationrecognition unit 112.

As described, by performing the determination processing according tothe example of FIG. 10, a walking state, a running state and anon-walking/non-running state can be distinguished.

(Jumping Recognition Method)

Next, a method of recognising whether a user is jumping or not will bedescribed with reference to FIG. 11. FIG. 11 is an explanatory diagramshowing a method of recognising whether a user is jumping or not.

First, when a user performs a behaviour, sensor data is input to themovement/state recognition unit 108. Here, three-axis directionacceleration data (x-acc, y-acc, z-acc) is input. When the sensor datais input, the movement/state recognition unit 108 calculates a jumpacceleration expressed by the magnitude of x-acc, y-acc and z-acc(S1200). Then, the movement/state recognition unit 108 removes, by aband-pass filter (BPF), a frequency outside a jumping recognition valuerange at which a user is recognised to be jumping (S1202). Then, themovement/state recognition unit 108 calculates an absolute value of avalue that passed through the BPF, and outputs the same as a compensatedjump acceleration (S1204). When an absolute value is obtained in thismanner, a noise component caused due to shaking, rocking or the like ofa housing occurring at the time of jumping is more removed than for thejump acceleration.

Then, the movement/state recognition unit 108 removes, from thecompensated jump acceleration, a frequency range at which jumping iseasily erroneously recognised, by using a low-pass filter (LPF) (S1206).Then, the movement/state recognition unit 108 calculates, from the datathat passed through the LPF, a jumping-state determination value (det)for determination of whether the user is jumping or not. Next, themovement/state recognition unit 108 determines whether the jumping-statedetermination value is larger than a minimum jumping recognition valueD4 which is the lower limit for jumping to be recognised (S1208). In acase it is larger than the minimum jumping recognition value D4, themovement/state recognition unit 108 determines that the user is jumping,and inputs information indicating jumping to the behaviour/situationrecognition unit 112 (S1210). On the other hand, in a case it is smallerthan the minimum jumping recognition value D4, the movement/staterecognition unit 108 determines that the user is not jumping, and inputsinformation indicating that the user is not jumping to thebehaviour/situation recognition unit 112 (S1212).

As described, by performing the determination processing according tothe example of FIG. 11, a jumping state and a non-jumping state can bedistinguished.

(Posture Changing Recognition Method)

Next, a method of recognising whether a user is sitting or standing willbe described with reference to FIG. 12. FIG. 12 is an explanatorydiagram showing a method of recognising whether a user is sitting orstanding. Additionally, to recognise whether a user is sitting orstanding is to recognise standing up of a user who is sitting or sittingdown of a user who is standing. That is, it is to recognise a change ina user's posture.

First, when a user performs a behaviour, sensor data is input to themovement/state recognition unit 108. Here, three-axis directionacceleration data (x-acc, y-acc, z-acc) is input. When the sensor datais input, the movement/state recognition unit 108 removes, from thex-acc, y-acc and z-acc, a frequency range at which a change in theuser's posture is easily erroneously recognised, by using a low-passfilter (LPF) (S1300). Then, the movement/state recognition unit 108calculates x-grav, y-grav and z-grav, based respectively on the x-acc,y-acc and z-acc. The x-grav, y-grav and z-grav are pieces of gravitydata indicating the direction of gravity.

Next, the movement/state recognition unit 108 calculates valuesδ(x-grav) indicating a change in the calculated x-grav, δ(y-grav)indicating a change in the calculated y-grav, and δ(z-grav) indicating achange in the calculated z-grav (S1302). Then, the movement/staterecognition unit 108 calculates a posture change value indicating themagnitude of the δ(x-grav), δ(y-grav) and δ(z-grav) (S1304). Then, themovement/state recognition unit 108 removes, from the calculated posturechange value, a range at which a change in the user's posture is easilyerroneously recognised, by using a low-pass filter (LPF) (S1306), andcalculates a posture change determination value (det) for determinationof whether the posture is changing or not.

Then, the movement/state recognition unit 108 determines whether or notthe posture change determination value is larger than a minimum posturechange recognition value D5 which is the lower limit for the posturechange of a user to be recognised (S1308). In a case the posture changedetermination value is smaller than D5, the movement/state recognitionunit 108 determines that there is no change in the posture, and inputs,to the behaviour/situation recognition unit 112, information indicatingthat there is no posture change (S1316). On the other hand, in a casethe posture change determination value is larger than D5, themovement/state recognition unit 108 proceeds to the process of stepS1310, and determines whether the user is sitting or standing (S1310).In a case the user was already standing, the movement/state recognitionunit 108 determines that the user sat down, and inputs, to thebehaviour/situation recognition unit 112, information indicating thesitting down (S1312). On the other hand, in a case the user was alreadyseated, the movement/state recognition unit 108 determines that the userstood up, and inputs, to the behaviour/situation recognition unit 112,information indicating the standing up (S1314).

As described, by performing the determination processing according tothe example of FIG. 12, presence or absence of posture change can bedistinguished.

(Recognition Method of Ascending/Descending in Elevator)

Next, a method of recognising whether a user is in an elevator or notwill be described with reference to FIG. 13. FIG. 13 is an explanatorydiagram showing a method of recognising whether a user is in an elevatoror not.

First, when a user performs a behaviour, sensor data is input to themovement/state recognition unit 108. Here, three-axis directionacceleration data (x-acc, y-acc, z-acc) is input. When the sensor datais input, the movement/state recognition unit 108 removes, from thex-acc, y-acc and z-acc, a frequency range at which acceleration in agravity direction is easily erroneously recognised, by using a low-passfilter (LPF) (S1400). Then, the movement/state recognition unit 108calculates gravity direction acceleration sensor data (acc) based on thex-acc, y-acc and z-acc that passed through the LPF (S1402).

Furthermore, the movement/state recognition unit 108 calculates andrecords gravity adjustment data expressed by the magnitude of the x-acc,y-acc and z-acc to enable adjustment of the value of gravity (S1404,S1406). Then, the movement/state recognition unit 108 reads a specificamount of the gravity adjustment data, and calculates a gravityadjustment variance (var) which is the variance of the gravityadjustment data (S1408). Furthermore, the movement/state recognitionunit 108 reads a specific amount of the gravity adjustment data, andcalculates gravity adjustment average data which is the average value ofthe gravity adjustment data (S1408).

Then, the movement/state recognition unit 108 determines whether or notthe gravity adjustment variance is smaller than a maximum allowablegravity adjustment variance V1 which is a maximum value that allowsadjustment of gravity (S1410). In a case the gravity adjustment varianceis larger than V1, the movement/state recognition unit 108 does notupdate the value of gravity (S1412). On the other hand, in a case thegravity adjustment variance is smaller than the maximum allowablegravity adjustment variance V1, the movement/state recognition unit 108determines whether or not the gravity adjustment average data is largerthan a minimum allowable gravity average value A1 which is a minimumaverage value that allows adjustment of gravity but smaller than amaximum allowable gravity average value A2 which is a maximum averagevalue that allows adjustment of gravity (S1414).

In a case the gravity adjustment average data is larger than A1 butsmaller than A2, the movement/state recognition unit 108 proceeds to theprocess of step S1418. On the other hand, in other cases, themovement/state recognition unit 108 does not update the value of thegravity (S1416). In a case it proceeded to the process of step S1418,the movement/state recognition unit 108 removes a low frequency range atwhich gravity is easily erroneously recognised, by using a low-passfilter (LPF) (S1418), and calculates compensated gravity adjustmentaverage data. Next, the movement/state recognition unit 108 calculates adifference between the gravity direction acceleration sensor data andthe compensated gravity adjustment average data (S1420). Then, themovement/state recognition unit 108 calculates elevatorascending/descending determination data by removing, from the calculateddifference, a frequency range at which a user is easily erroneouslyrecognised to be in an elevator (S1422).

Next, the movement/state recognition unit 108 determines whether or notthe elevator ascending/descending determination data is larger than aspecific value D6 (S1424). In a case the elevator ascending/descendingdetermination data is larger than D6, the movement/state recognitionunit 108 proceeds to the process of step S1426. On the other hand, in acase the elevator ascending/descending determination data is smallerthan the specific value D6, the movement/state recognition unit 108proceeds to the process of step S1432. Here, the specific value D6 isthe lower limit at which it is possible to recognise the start ofascending of a user in an elevator.

In a case it proceeded to the process of step S1426, the movement/staterecognition unit 108 determines whether or not the elevatorascending/descending determination data has exceeded the specific valueD6 for the first time (S1426). In a case it is the first time, themovement/state recognition unit 108 proceeds to the step of S1428 anddetermines that the elevator is ascending, and inputs informationindicating ascending in an elevator to the behaviour/situationrecognition unit 112 (S1428). On the other hand, in a case it is not thefirst time, the movement/state recognition unit 108 proceeds to theprocess of step S1430 and determines that descending in the elevator hasended, and inputs information indicating the end of descending in anelevator to the behaviour/situation recognition unit 112 (S1430).

In a case it proceeded to the process of step S1432, the movement/staterecognition unit 108 determines whether or not the elevatorascending/descending determination data is larger than a specific valueD7 (S1432). Here, the specific value D7 is the upper limit at which itis possible to recognise the start of descending of a user in anelevator. In a case the elevator ascending/descending determination datais larger than the specific value D7, the movement/state recognitionunit 108 proceeds to the process of step S1434. On the other hand, in acase the elevator ascending/descending determination data is smallerthan the specific value D7, the movement/state recognition unit 108proceeds to the process of step S1440.

In a case it proceeded to the process of step S1434, the movement/staterecognition unit 108 determines whether or not the elevatorascending/descending determination data has fallen below the specificvalue D7 for the first time (S1434). In a case it is the first time, themovement/state recognition unit 108 proceeds to the step of S1436 anddetermines that the elevator is descending, and inputs informationindicating descending in an elevator to the behaviour/situationrecognition unit 112 (S1436). On the other hand, in a case it is not thefirst time, the movement/state recognition unit 108 proceeds to theprocess of step S1438 and determines that ascending in the elevator hasended, and inputs information indicating the end of ascending in anelevator to the behaviour/situation recognition unit 112 (S1438).

In a case it proceeded to the process of step S1440, the movement/staterecognition unit 108 determines whether or not the user is currently inan elevator (S1440). In a case the user is in an elevator, themovement/state recognition unit 108 proceeds to the process of stepS1442 and determines that the elevator is not in a state of accelerationor deceleration, and inputs information indicating a state of noacceleration or deceleration of elevator to the behaviour/situationrecognition unit 112 (S1442). On the other hand, in a case the user isnot in an elevator, the movement/state recognition unit 108 proceeds tothe process of step S1444, and inputs information indicating a statewhere the user is not in an elevator to the behaviour/situationrecognition unit 112 (S1444).

As described, by performing the determination processing according tothe example of FIG. 13, ascending or descending in an elevator can bedistinguished.

(Recognition Method of Riding on Train)

Next, a method of recognising whether a user is riding on a train or notwill be described with reference to FIG. 14. FIG. 14 is an explanatorydiagram showing a method of recognising whether a user is riding on atrain or not.

First, when an user performs a behaviour, sensor data is input to themovement/state recognition unit 108. Here, three-axis directionacceleration data (x-acc, y-acc, z-acc) is input. When the sensor datais input, the movement/state recognition unit 108 removes, from thex-acc, y-acc and z-acc, a frequency range at which a user is easilyerroneously recognised to be riding on a train, by using a low-passfilter (LPF) (S1500). Then, the movement/state recognition unit 108calculates horizontal direction acceleration data and vertical directionacceleration data based on the x-acc, y-acc and z-acc from which thefrequency range described above has been removed (S1502, S1504). Here,the horizontal direction and the vertical direction respectively mean adirection horizontal or vertical to the ground on which the train isrunning.

Next, the movement/state recognition unit 108 records, in FIFO manner,specific amounts of the horizontal direction acceleration data and thevertical direction acceleration data (S1506, S1508). Then, themovement/state recognition unit 108 reads a specific amount of thehorizontal direction acceleration data, and calculates a horizontaldirection variance (h-var) which is the variance of the horizontaldirection acceleration data (S1510). Also, the movement/staterecognition unit 108 reads a specific amount of the vertical directionacceleration data, and calculates a vertical direction variance (v-var)which is the variance of the vertical direction acceleration data(S1512). The horizontal direction variance (h-var) indicates the degreeof rocking and rolling in the horizontal direction detected at the timeof a train running. Also, the vertical direction variance (v-var)indicates the degree of rocking and pitching in the vertical directiondetected at the time of the train running.

Then, the movement/state recognition unit 108 determines whether or notthe vertical direction variance (v-var) is larger than a minimumallowable vertical variance V1 which is a minimum vertical directionvariance that is allowed but smaller than a maximum allowable verticalvariance V2 which is a maximum vertical variance that is allowed(S1514). In a case the vertical direction variance (v-var) is V1 or lessor V2 or more, the movement/state recognition unit 108 sets train-ridedetermination data (det) to zero (S1528). On the other hand, in a casethe vertical direction variance is larger than V1 but smaller than V2,the movement/recognition unit 108 proceeds to the process of step S1516.

In a case it proceeded to the process of step S1516, the movement/staterecognition unit 108 determines which of the vertical direction varianceand the horizontal direction variance is smaller (S1516). In a case thevertical direction variance (v-var) is smaller, the movement/staterecognition unit 108 integrates the vertical direction variance (v-var)for a specific amount of data, and calculates an integral (S1518). Onthe other hand, in a case the horizontal direction variance (h-var) issmaller, the movement/state recognition unit 108 integrates thehorizontal direction variance (h-var) for a specific amount of data, andcalculates an integral (S1520). Then, the integrals obtained by theprocesses of steps S1518 and S1520 are set as the train-ridedetermination data (det) which is for determination of whether a user isriding on a train or not.

Then, the movement/state recognition unit 108 determines whether or notthe train-ride determination data is larger than a minimum train-riderecognition value D8 which is the lower limit at which a user isrecognised to be riding on a train (S1522). In a case the train-ridedetermination data is larger than D8, the movement/state recognitionunit 108 determines that the user is riding on a train, and inputsinformation indicating a state where the user is riding on a train tothe behaviour/situation recognition unit 112 (S1524). On the other hand,in a case the train-ride determination data is smaller than D8, themovement/state recognition unit 108 determines that the user is notriding on a train, and inputs information indicating that the user isnot riding on a train to the behaviour/situation recognition unit 112(S1526).

As described, by performing the determination processing according tothe example of FIG. 14, whether or not it is a state where a user isriding on a train can be distinguished. By focusing on the state of atrain from acceleration to deceleration, a case where the user is ridingon a train stopped at a station, a case where the train stopped at astation, a case where the user got off a train that arrived at a stationand started walking, and the like, can also be distinguished. Thesedetermination results may be notified to the behaviour/situationrecognition unit 112.

(Right-Turn/Left-Turn Recognition Method)

Next, a method of recognising a right turn or a left turn of a user willbe described with reference to FIG. 15. FIG. 15 is an explanatorydiagram showing a method of recognising a right turn or a left turn of auser.

First, when a user performs a behaviour, sensor data is input to themovement/state recognition unit 108. Here, three-axis directionacceleration data (x-acc, y-acc, z-acc) and gyro data (x-gyro, y-gyro,z-gyro) are input. When the sensor data is input, the movement/staterecognition unit 108 removes, from the input sensor data, a frequencyrange at which a right turn or a left turn is easily erroneouslyrecognised, by using a low-pass filter (LPF) (S1600). Then, themovement/state recognition unit 108 calculates an angular velocity in agravity direction based on the x-acc, y-acc and z-acc from which thefrequency range described above has been removed and the x-gyro, y-gyroand z-gyro (S1602).

Next, the movement/state recognition unit 108 calculates a compensatedangular velocity (det) by removing, from the calculated angular velocityby using a band-pass filter (BPF), a value outside a turn recognitionrange for recognition of a right turn or a left turn (S1604). Then, themovement/state recognition unit 108 determines whether the compensatedangular velocity is smaller than a maximum right-turn recognition valueD9 which is the upper limit for recognition of a right turn of a user(S1606). In a case the compensated angular velocity is smaller than D9,the movement/state recognition unit 108 determines that the user isturning right, and inputs the determination result to thebehaviour/situation recognition unit 112 (S1608). On the other hand, ina case the compensated angular velocity is D9 or more, themovement/state recognition unit 108 proceeds to the process of stepS1610.

In a case it proceeded to the process of step S1610, the movement/staterecognition unit 108 determines whether or not the compensated angularvelocity is larger than a minimum left-turn recognition value D10 whichis the lower limit for recognition of a left turn of the user (S1610).In a case the compensated angular velocity is larger than D10, themovement/state recognition unit 108 determines that the user is turningleft, and inputs the determination result to the behaviour/situationrecognition unit 112 (S1612). On the other hand, in a case thecompensated angular velocity is smaller than D10, the movement/staterecognition unit 108 determines that the user is turning neither rightnor left, and inputs the determination result to the behaviour/situationrecognition unit 112 (S1614).

As described, by performing the determination processing according tothe example of FIG. 15, a right turn and a left turn of a user can bedistinguished.

Heretofore, the function of the movement/state recognition unit 108 hasbeen described in detail. As described above, a movement/state patterndoes not indicate a specific daily behaviour of a user. Themovement/state pattern can be said to express a state of a user at acertain moment (a short period of time). Thus, even if the records ofthe movement/state patterns are accumulated and the pieces ofinformation corresponding to one day are lined up, it is difficult tolook back on a day's events without resorting to one's memory. Forexample, even when referring to pieces of information such as “walking”for 10 minutes, “staying still” for 30 seconds, “running” for 3 minutes,“staying still” for 10 seconds, “turning right,” “riding on a train” for15 minutes, “walking” for 10 minutes, “turning right,” . . . , it isextremely difficult to know what one has done at which place. For thisreason, means for detecting a more specific daily behaviour (HCbehaviour) is wanted.

<1-3: Function of Geo-Categorisation Unit 110>

Next, a function of the geo-categorisation unit 110 will be described indetail with reference to FIGS. 16 to 19. FIGS. 16 to 19 are explanatorydiagrams for describing a function of the geo-categorisation unit 110.

First, reference will be made to FIG. 16. As described above, thegeo-categorisation unit 110 selects a geo category code (or informationbased thereon) corresponding to location information on the currentlocation input from the location sensor 104. At this time, thegeo-categorisation unit 110 acquires map information MP and geo categoryinformation GC from a map database, and detects an attribute of abuilding or the like at the current location. The map database isregistered with information such as (A1) map information, (A2) shapeelement data (information on the shape of a building, a site or a road),and (A3) information on a store (an occupational category) or the likeregistered with the building or the site. The (A1) and (A2) correspondto the map information MP, and (A3) corresponds to the geo categoryinformation GC. Additionally, the map database does not have to beincluded in the behaviour/situation analysis system 10, and a mapdatabase published on a Web may be alternatively used, for example.

As shown in FIG. 17, in the geo category information GC, buildings orthe like are classified into categories according to specific categorytypes, and each category is assigned with a category code (a geocategory code). Also, as shown in FIG. 17, the geo category may beclassified into a major category, a middle category and a minor category(not shown). Furthermore, in the example of FIG. 17, the category codesshown in the major category column and the middle category columnindicate the geo category codes. For example, in a case a batting centreis detected at the current location by the map information MP, thegeo-categorisation unit 110 outputs a geo category code 1305000.

Furthermore, the geo-categorisation unit 110 detects geo category codesof buildings or the like existing in the vicinity of the currentlocation, and creates a histogram. However, in a case the surroundingenvironment of the current location is not to be taken intoconsideration in the behaviour/situation pattern detection process, thecreation of the histogram can be omitted. In a case the surroundingenvironment of the current location is to be taken into consideration,the geo-categorisation unit 110 acquires a geo category code group ofthe buildings or the like existing in the vicinity of the currentlocation by using the map information MP and the geo categoryinformation GC. Then, the geo-categorisation unit 110 tallies the numberof buildings or the like for each geo category, and creates a histogramof geo categories (hereinafter, geo category histogram) as shown in FIG.18.

As shown in FIG. 19, the geo category code expresses the environment ofthe current location in a pinpoint manner, and the geo categoryhistogram expresses the surrounding environment of the current location.For example, in a case the geo category code indicates a train station,the behaviour of a user at the time point of acquisition of the locationinformation on the current location is narrowed down to behaviour thatcan be performed within a station. On the other hand, in a case the geocategory code indicates a public road, it is difficult to narrow downthe behaviour of the user based on the geo category code. However, if itis seen from the geo category histogram that the user is in a regionwhere there are many retail stores, the behaviour of the user can bemore narrowed down even if the geo category code indicates the samepublic road.

For this reason, by using the geo category histogram together with thegeo category code, the accuracy of behaviour/situation pattern detectiondescribed later can be improved. Accordingly, the geo-categorisationunit 110 calculates the geo category histogram (B2) together with thegeo category code (B1), and inputs the same to the behaviour/situationrecognition unit 112. Furthermore, the geo-categorisation unit 110 alsoinputs information (B3) on the latitude and longitude indicating thecurrent location, the amount of movement or the like to thebehaviour/situation recognition unit 112.

Heretofore, the function of the geo-categorisation unit 110 has beendescribed. Additionally, the location information on the currentlocation may indicate a representative point obtained by clusteringmultiple pieces of location information.

<1-4: Function of Behaviour/Situation Recognition Unit 112>

Next, a function of the behaviour/situation recognition unit 112 will bedescribed in detail with reference to FIGS. 20 to 30B. FIGS. 20 to 30Bare explanatory diagrams for explaining a function of thebehaviour/situation recognition unit 112.

(Overview)

First, reference will be made to FIG. 20. As described above, thetime/calendar information, the movement/state pattern and the geocategory code are input to the behaviour/situation recognition unit 112.Furthermore, information on the movement/state feature quantity used fordetection of the movement/state pattern, the sensor data, the geocategory histogram obtained from the geo category codes, the amount ofmovement, the latitude and longitude, movement speed, or the like, isinput to the behaviour/situation recognition unit 112. Also, personalprofile information or a feedback from a user may be input to thebehaviour/situation recognition unit 112. When these pieces ofinformation are input, the behaviour/situation recognition unit 112detects a behaviour/situation pattern that matches the combination ofthe input pieces of information. At this time, the behaviour/situationrecognition unit 112 detects the behaviour/situation pattern based onthe rule-based determination or the learning model determination.

(Rule-Based Determination; Geo Category Code)

First, a behaviour/situation pattern detection method based on therule-based determination will be described. Additionally, adetermination method based on the geo category code will be describedhere. As described above, the score map SM is used for the rule-baseddetermination. Here, a score map SM as shown in FIG. 21 is assumed.

In the example of FIG. 21, a score is assigned to a combination of amiddle-category geo category code and a behaviour/situation pattern. Inthe example of FIG. 21, the behaviour/situation patterns to be takeninto consideration are “sport,” “walk,” “recreation,” “shopping,” . . ., “work,” “viewing,” and “sleeping.” However, the types ofbehaviour/situation patterns are not limited to the above, and variousbehaviour/situation patterns as shown in FIG. 26 can be taken intoconsideration, for example. The behaviour/situation patterns to be takeninto consideration may be selected in advance by the user, or thoseappropriate for the user may be automatically selected by adetermination model created by using a machine learning algorithm.

Furthermore, the score map SM as shown in FIG. 21 is provided for eachcombination of a type of the time/calendar information and a type of themovement/state pattern. Therefore, there are multiple score maps SM. Asshown in FIG. 22, first, the behaviour/situation recognition unit 112selects, from the multiple score maps SM, combinations of scores eachcorresponding to the input geo category code (S10). In a case a geocategory code 1905000 is input to the behaviour/situation recognitionunit 112 in the example of FIG. 21, a group of scores “meal=1, study=2,work=4” is selected for each type of the score maps SM. That is,multiple combinations of score groups are selected.

Next, the behaviour/situation recognition unit 112 selects types ofscore maps SM corresponding to the input movement/state pattern (S12).At this time, the behaviour/situation recognition unit 112 extractsscore groups corresponding to the selected types of the score maps SM.Then, the behaviour/situation recognition unit 112 selects, from thetypes of the score maps SM selected in step S12, a type of the score mapSM corresponding to the input time/calendar information (S14). At thistime, the behaviour/situation recognition unit 112 extracts a scoregroup corresponding to the selected type of the score map SM. As aresult, a score group corresponding to the movement/state pattern, thetime/calendar information and the geo category code that have been inputis extracted.

In the process of step S12 described above, score maps SM correspondingto a movement/state pattern is selected. This selection process isrealised by the operation described in FIG. 23. As illustrated in FIG.23, score map groups P1 to P9 each formed from multiple score maps SMare associated with each movement/state pattern. Accordingly, thebehaviour/situation recognition unit 112 distinguishes an inputmovement/state pattern (and movement speed/the amount of movement) basedon a specific determination condition, and selects a score map groupcorresponding to the determination result. For example, in a case themovement/state pattern is “walking continued for more than a time T1,” ascore map group P5 is selected.

As described above, when a score group corresponding to the combinationof the movement/state pattern, the time/calendar information and the geocategory code that have been input is extracted, the behaviour/situationrecognition unit 112 detects the highest score among the extracted scoregroup. Then, the behaviour/situation recognition unit 112 specifies abehaviour/situation pattern corresponding to the highest score, andoutputs the specified behaviour/situation pattern. Thebehaviour/situation pattern output by the behaviour/situationrecognition unit 112 is used for the provision of the recommendedservice SV or is used by the application AP.

Moreover, the behaviour/situation recognition unit 112 may be configuredto use not only the score group corresponding to the current input, butalso a score group corresponding to past input, and to specify thebehaviour/situation pattern by using HMM or the like.

Heretofore, the behaviour/situation pattern detection method based onthe rule-based determination that uses the geo category code has beendescribed.

(Rule-Based Determination; Geo Category Histogram)

Next, a behaviour/situation pattern detection method based on therule-based determination that uses the geo category histogram will bedescribed with reference to FIG. 24. In a case of using the geo categoryhistogram, the geo category codes of buildings or the like existing inthe vicinity of the current location and the histogram of the geocategories are input to the behaviour/situation recognition unit 112.Here, it is assumed that ten geo categories (GC1, . . . , GC10) aredetected in the vicinity of the current location.

When the geo category codes are input, the behaviour/situationrecognition unit 112 extracts a score group corresponding to eachcategory code from each score map SM. In the example of FIG. 24, geocategory codes 2303000, 1905000, . . . , 1602000 corresponding to thegeo categories GC1, GC2, . . . , GC10 are input, and respectivecorresponding score groups are extracted. When the score groups areextracted, the behaviour/situation recognition unit 112 calculates scoredistributions pd1, pd2, . . . , pd10 by normalising each of the scoregroups. This normalisation is performed by using the highest scoreincluded in each score group.

Next, the behaviour/situation recognition unit 112 performsmultiplication by values of the input geo category histogram on thecalculated score distributions pd1, pd2, . . . , pd10. For example,multiplication is performed on the score distribution pd1 by a histogramvalue 0 of the geo category GC1. Also, multiplication is performed onthe score distribution pd2 by a histogram value 10 of the geo categoryGC2. Similarly, multiplication is performed on the score distributionspd3, . . . , pd10 respectively by histogram values 45, . . . , 20 of thegeo categories GC3, . . . , GC10. Next, the behaviour/situationrecognition unit 112 totals the score distributions pd1, . . . , pd10that have been weighted by the histogram values as above for each typeof the score maps SM, and calculates score distributions PD1, . . . ,PDn for each score map SM. Here, the n indicates the number ofcombinations of the time/calendar information and the movement/statepattern.

Next, the behaviour/situation recognition unit 112 specifies the type ofthe score map SM corresponding to the combination of the time/calendarinformation and the movement/state pattern that has been input (forexample, the k-th score map SM), and selects a score distribution PDkcorresponding to the specified type. Then, the behaviour/situationrecognition unit 112 detects the highest score among the selected scoredistribution PDk, and outputs a behaviour/situation patterncorresponding to the highest score detected. According to suchconfiguration, detection of a behaviour/situation pattern that takesinto consideration the surrounding environment of the current locationcan be performed. Additionally, a method of selecting a geo categorycode whose histogram value is the largest and detecting abehaviour/situation pattern by using the selected geo category code canalso be conceived. In the case of this method, a behaviour/situationpattern corresponding to “meal” is detected with high probability in anarea where there are many restaurants.

Additionally, in the above example, a method of detecting the highestscore has been shown, but however, in a case of selecting multiplecandidates for the behaviour/situation pattern, for example, instead ofthe highest score, the behaviour/situation patterns to be the candidatesmay be detected in order from the highest score. In this case,appropriate behaviour/situation patterns are narrowed down from thedetected candidates for the behaviour/situation pattern based on theuser's profile information, history of past behaviour/situation patterndetection or the like. This configuration is an example, and suchmodified example is, of course, included in the technical scope of thepresent embodiment.

Heretofore, the behaviour/situation pattern detection method based onthe rule-based determination that uses the geo category histogram hasbeen described. Additionally, the method that uses the geo category codeand the method that uses the geo category histogram may also be used inparallel. By using these in parallel, an appropriate behaviour/situationpattern that takes into consideration the environment of the place theuser is at and the surrounding environment (atmosphere) of the place canbe detected in a pinpoint manner.

(Learning Model Determination; Geo Category Code)

Next, a behaviour/situation pattern detection method based on thelearning model determination will be described with reference to FIG.25. As described above, according to the learning model determination, adetermination model is created by using a machine learning algorithm,and a behaviour/situation pattern is detected by using the createddetermination model. As the machine learning algorithm, linearregression, nonlinear regression, SVM, Boosting and the like are used.Furthermore, a feature quantity selection process by a genetic searchmethod may be combined with the machine learning algorithm.

Furthermore, a feature vector given as teacher data at the time ofcreating a determination model includes, for example, the time/calendarinformation (e.g., date, time, day of the week, or holiday/non-holiday),the movement/state pattern, the movement/state feature quantity, thesensor data (the acceleration data, the gyro data, or the like), the geocategory code, the geo category histogram (the number per code), thelatitude and longitude, and the like. Furthermore, as the featurevector, any detection data relating to the behaviour of a user and itsprocessed data can be used. Additionally, a response variable given asthe teacher data is correct data indicating the correctbehaviour/situation pattern. Moreover, by using, as the teacher data,data selectively picked from data of a group of people that may takesimilar behaviours, for example, “students,” “members of society,”“males,” and “females,” a determination model that is optimised for eachgroup can be created.

The mechanism of machine learning is broadly to prepare a large numberof feature vectors for which the correct data is known, to apply thefeature vectors to pairs of functions selected from a specific functiongroup, and to extract a pair of functions from which the same featurequantity (answer data) can be obtained when multiple feature vectorscorresponding to the same correct data are applied to the pair offunctions. The specific function group includes any function (algorithm)such as differential operation output, maximum value output, low-passfiltering, unbiased variance output, and Fourier transform output. Thatis, an algorithm (determination model) capable of combining thesefunctions and outputting correct data with high accuracy isautomatically created.

The determination model created in this manner is expected to output acorrect or almost correct behaviour/situation pattern for a featurevector of the same format that is arbitrarily input. Thus, thebehaviour/situation recognition unit 112 inputs, to the createddetermination model, a feature vector formed from sensor data or thelike actually observed, and detects a behaviour/situation pattern. Ifsufficient learning has been performed, a behaviour/situation patterncan be detected by this method with high accuracy. However, the processof creating a determination model by a machine learning algorithm is aprocess for which the amount of computation is extremely large.Therefore, as has been described with reference to FIGS. 2 to 6, asystem configuration has to be modified in a case of using the learningmodel determination. Furthermore, a method of using the rule-baseddetermination and the learning model determination in combination canalso be conceived.

Heretofore, the behaviour/situation pattern detection method that usesthe learning model determination has been described. As described above,when using the learning model determination, a behaviour/situationpattern can be detected with high accuracy if sufficient learning hasbeen performed. Also, by rebuilding the determination model by taking afeedback from a user into consideration, a determination model capableof detecting a behaviour/situation pattern with further improvedaccuracy can be created. Accordingly, using the learning modeldetermination is beneficial for improving the accuracy ofbehaviour/situation pattern detection.

(Operational Flow)

Next, an operational flow of the behaviour/situation recognition unit112 relating to the behaviour/situation pattern detection method will bedescribed with reference to FIGS. 27 to 30B. Note that FIG. 27 is anexplanatory diagram showing an overall operational flow of thebehaviour/situation analysis system 10.

(Overall System)

First, an overall operational flow of the behaviour/situation analysissystem 10 relating to the behaviour/situation pattern detection methodwill be described with reference to FIG. 27.

As shown in FIG. 27, when a user performs some behaviour (S20), sensordata is acquired by the motion sensor 102 (S22). Also, the locationinformation on the current location is acquired by the location sensor104. Then, the sensor data is input to the movement/state recognitionunit 108, and the location information on the current location is inputto the geo-categorisation unit 110. Next, a movement/state pattern isdetected by the movement/state recognition unit 108, and a geo categorycode (histogram) is extracted by the geo-categorisation unit 110 (S24).Then, information on the movement/state pattern, the geo category code(histogram) and the like are input to the behaviour/situationrecognition unit 112.

When the information on the movement/state pattern, the geo categorycode (histogram) and the like are input, the behaviour/situationrecognition unit 112 detects a behaviour/situation pattern by usingthese pieces of information (S26). Additionally, the process of step S26will be described later in detail. When a behaviour/situation pattern isdetected by the behaviour/situation recognition unit 112, information onthe detected behaviour/situation pattern is input to an application orthe like (S28). Then, a recommended service SV is provided by using thebehaviour/situation pattern, or a function corresponding to thebehaviour/situation pattern is provided to the user by the application.In the following, the process flow of step S26 will be described indetail.

(A: Rule-Based Determination)

Here, an operational flow of the behaviour/situation recognition unit112 relating to the behaviour/situation pattern detection method basedon the rule-based determination will be described with reference to FIG.28.

As shown in FIG. 28, first, the behaviour/situation recognition unit 112determines whether a user profile is registered or not (S102). In a casea user profile is registered, the behaviour/situation recognition unit112 proceeds to the process of step S104. On the other hand, in a case auser profile is not registered, the behaviour/situation recognition unit112 proceeds to the process of step S108.

In a case it proceeded to the process of step S104, thebehaviour/situation recognition unit 112 determines whether or not thelatitude and longitude of the current location that is input is home orworkplace (S104). For example, the behaviour/situation recognition unit112 refers to the user profile that is registered, and determineswhether or not the latitude and longitude of the current location matchthe latitude and longitude of home or workplace described in the userprofile. In a case neither home nor workplace exists at the location ofthe latitude and longitude, the behaviour/situation recognition unit 112proceeds to the process of step S110. On the other hand, in a case homeor workplace exists at the location of the latitude and longitude, thebehaviour/situation recognition unit 112 proceeds to the process of stepS106.

In a case it proceeded to the process of step S106, thebehaviour/situation recognition unit 112 selects score maps SM (scoregroups) corresponding to home or workplace (S106), and proceeds to theprocess of step S112. Furthermore, in a case it proceeded to the processof step S108 by the determination process of step S102, thebehaviour/situation recognition unit 112 selects a score group based ona geo category code (histogram) that is input (S108), and proceeds tothe process of step S112.

Also, in a case it proceeded to the process of step S110 by thedetermination process of step S104, the behaviour/situation recognitionunit 112 selects a score group based on the geo category code(histogram). Furthermore, the behaviour/situation recognition unit 112performs weighting for the selected score group based on the userprofile (S110), and proceeds to the process of step S112. For example,in a case it is described in the user profile that the user likesbaseball, the behaviour/situation recognition unit 112 performsweighting in such a way that the score of a behaviour/situationpattern=“baseball” becomes high.

In a case it proceeded to the process of step S112, thebehaviour/situation recognition unit 112 narrows down the types of thescore maps SM based on the movement/state pattern and the time/calendarinformation (S112). Next, the behaviour/situation recognition unit 112detects the highest score from a score group corresponding to the typewhich has been narrowed down by the process of step S112, and selects abehaviour/situation pattern corresponding to the highest score which hasbeen detected (S114). Then, the selected behaviour/situation pattern isinput to an application or the like, and the series of operationsrelating to the detection of a behaviour/situation pattern is ended.Additionally, in a case of using the geo category histogram, a scoredistribution is calculated in step S114, and a behaviour/situationpattern corresponding to the highest probability is selected.

Heretofore, the behaviour/situation pattern detection method based onthe rule-based determination has been described. Moreover, a method ofselecting a behaviour/situation pattern corresponding to the highestscore is used in the example of FIG. 28, but the behaviour/situationrecognition unit 112 may be configured to use not only the score groupcorresponding to the current input, but also a score group correspondingto past input, and to select the behaviour/situation pattern by usingHMM or the like.

(B: Learning Model Determination)

Next, an operational flow of the behaviour/situation recognition unit112 relating to the behaviour/situation pattern detection method basedon the learning model determination will be described with reference toFIG. 29.

As shown in FIG. 29, first, the behaviour/situation recognition unit 112determines whether a user profile is registered or not (S122). In a casea user profile is registered, the behaviour/situation recognition unit112 proceeds to the process of step S124. On the other hand, in a case auser profile is not registered, the behaviour/situation recognition unit112 proceeds to the process of step S126.

In a case it proceeded to the process of step S124, thebehaviour/situation recognition unit 112 selects a determination modelcreated by a machine learning algorithm with the user profile taken intoconsideration (S124), and proceeds to the process of step S128. On theother hand, in a case it proceeded to the process of step S126, thebehaviour/situation recognition unit 112 selects a general-purposedetermination model created by a machine learning algorithm without theuser profile taken into consideration (S126), and proceeds to theprocess of step S128.

In a case it proceeded to the process of step S128, thebehaviour/situation recognition unit 112 inputs, to the determinationmodel selected in step S124 or step S126, information (a feature vector)which has been input, and detects a behaviour/situation pattern matchingthe input feature vector (S128). Next, the behaviour/situationrecognition unit 112 outputs the behaviour/situation pattern detected bythe process of step S128 (S130). Then, the behaviour/situation patternthat is output is input to an application or the like, and the series ofoperations relating to the detection of a behaviour/situation pattern isended.

Heretofore, the behaviour/situation pattern detection method based onthe learning model determination has been described.

(Combined Usage)

Next, an operational flow of the behaviour/situation recognition unit112 relating to the behaviour/situation pattern detection method thatuses the rule-based determination and the learning model determinationin combination will be described with reference to FIGS. 30A and 30B.

As shown in FIG. 30A, first, the behaviour/situation recognition unit112 determines whether a user profile is registered or not (S142). In acase a user profile is registered, the behaviour/situation recognitionunit 112 proceeds to the process of step S144. On the other hand, in acase a user profile is not registered, the behaviour/situationrecognition unit 112 proceeds to the process of step S148.

In a case it proceeded to the process of step S144, thebehaviour/situation recognition unit 112 determines whether or not thelatitude and longitude of the current location that is input is home orworkplace (S144). For example, the behaviour/situation recognition unit112 refers to the user profile that is registered, and determineswhether or not the latitude and longitude of the current location matchthe latitude and longitude of home or workplace described in the userprofile. In a case neither home nor workplace exists at the location ofthe latitude and longitude, the behaviour/situation recognition unit 112proceeds to the process of step S150. On the other hand, in a case homeor workplace exists at the location of the latitude and longitude, thebehaviour/situation recognition unit 112 proceeds to the process of stepS146.

In a case it proceeded to the process of step S146, thebehaviour/situation recognition unit 112 selects score maps SM (scoregroups) corresponding to home or workplace (S146), and proceeds to theprocess of step S152. Furthermore, in a case it proceeded to the processof step S148 by the determination process of step S142, thebehaviour/situation recognition unit 112 selects a score group based ona geo category code (histogram) that is input (S148), and proceeds tothe process of step S152.

Also, in a case it proceeded to the process of step S150 by thedetermination process of step S144, the behaviour/situation recognitionunit 112 selects a score group based on the geo category code(histogram). Furthermore, the behaviour/situation recognition unit 112performs weighting for the selected score group based on the userprofile (S150), and proceeds to the process of step S152. For example,in a case it is described in the user profile that the user likes hotsprings, the behaviour/situation recognition unit 112 performs weightingin such a way that the score of a behaviour/situation pattern=“hotspring” becomes high.

In a case it proceeded to the process of step S152, thebehaviour/situation recognition unit 112 narrows down the types of thescore maps SM based on the movement/state pattern and the time/calendarinformation (S152). Next, the behaviour/situation recognition unit 112detects the highest score from a score group corresponding to the typewhich has been narrowed down by the process of step S152, and selects abehaviour/situation pattern corresponding to the highest score which hasbeen detected (S154). Additionally, in a case of using the geo categoryhistogram, a score distribution is calculated in step S154, and abehaviour/situation pattern corresponding to the highest probability isselected.

Next, the behaviour/situation recognition unit 112 proceeds to step S156shown in FIG. 30B, and determines whether or not the highest scoredetected in step S154 is a specific value or more (S156). In a case thehighest score is a specific value or more, the behaviour/situationrecognition unit 112 proceeds to the process of step S166. On the otherhand, in a case it is not a specific value or more, thebehaviour/situation recognition unit 112 proceeds to the process of stepS158.

In a case it proceeded to the process of step S158, thebehaviour/situation recognition unit 112 determines whether a userprofile is registered or not (S158). In a case a user profile isregistered, the behaviour/situation recognition unit 112 proceeds to theprocess of step S160. On the other hand, in a case a user profile is notregistered, the behaviour/situation recognition unit 112 proceeds to theprocess of step S162.

In a case it proceeded to the process of step S160, thebehaviour/situation recognition unit 112 selects a determination modelcreated by a machine learning algorithm with the user profile taken intoconsideration (S160), and proceeds to the process of step S164. On theother hand, in a case it proceeded to the process of step S162, thebehaviour/situation recognition unit 112 selects a general-purposedetermination model created by a machine learning algorithm without theuser profile taken into consideration (S162), and proceeds to theprocess of step S164.

In a case it proceeded to the process of step S164, thebehaviour/situation recognition unit 112 inputs, to the determinationmodel selected in step S160 or step S162, information (a feature vector)which has been input, and detects a behaviour/situation pattern matchingthe input feature vector (S164). Next, the behaviour/situationrecognition unit 112 outputs the behaviour/situation pattern detected bythe process of step S164 or the behaviour/situation pattern selected instep S154 in FIG. 30A (S166). Then, the behaviour/situation pattern thatis output is input to an application or the like, and the series ofoperations relating to the detection of a behaviour/situation pattern isended.

Heretofore, an operational flow of the behaviour/situation recognitionunit 112 relating to the behaviour/situation pattern detection methodthat uses the rule-based determination and the learning modeldetermination in combination has been described. Additionally, abehaviour/situation pattern corresponding to the highest score isselected by the process of step S154 in the example of FIG. 30A, butother method can also be used as the method of selecting abehaviour/situation pattern. For example, a method can be used that usesa score group corresponding to past input in addition to the score groupcorresponding to the current input, and that selects thebehaviour/situation pattern by using HMM or the like.

As has been described, by using the behaviour/situation patterndetection method according to the present embodiment, it becomespossible to detect a behaviour/situation pattern relating to a user'sdaily behaviour (HC behaviour) as illustrated in FIG. 26. As a result,it becomes possible to use a user's daily behaviour which is hard topredict from a behaviour history expressed by an accumulation of LCbehaviours.

2: Second Embodiment

Next, the second embodiment of the present invention will be described.The present embodiment relates to a method of using abehaviour/situation pattern obtained by the behaviour/situation patterndetection method described in the first embodiment described above.Particularly, the technology of the present embodiment relates to amethod of correlating schedule information registered by a user and abehaviour/situation pattern that is detected with each other, andproviding the user with appropriate information in accordance with thesituation.

<2-1: Overview of System>

First, an overview of a function realised by a behaviour/situationanalysis system 20 according to the present embodiment will be describedwith reference to FIGS. 31 and 32. FIGS. 31 and 32 are explanatorydiagrams showing an effective method of presenting information based oncorrelation with a schedule application.

First, reference will be made to FIG. 31. In the example of FIG. 31,schedule information indicating a timetable of a trip is registered in aschedule application, and the contents of notification information isdecided by combining the registered information and a result ofbehaviour/situation detection. For example, consideration will be givento information to be notified at the time point of a user walking to thestation in the morning. Additionally, “walking” is described as theschedule information in the example of FIG. 31, but even without thedescription, a situation where the user is heading for the station inthe morning can be detected from the description of “train” andinformation on the time. Moreover, the schedule information may beacquired via a network.

First, it can be presumed, from the information “train, 9:00-10:00”registered in the schedule information, that a situation will arisewhere a user heads for the station before 9:00. However, it is difficultto estimate, from the schedule information, the timing of notificationof information that is to be notified at the time of the user headingfor the station. However, according to the present embodiment, abehaviour/situation pattern of a user can be detected, and thus anotification timing can be decided by using the detectedbehaviour/situation pattern. For example, train guide may be notified ata timing of detection of a behaviour/situation pattern “moving(walking).”

Similarly, by using a behaviour/situation pattern, a situation where theuser is almost arriving at a tourist spot, a situation where the user isdoing sightseeing around the tourist spot, a situation where thesightseeing is almost over, a situation where the user is on the wayhome, and the like, can be recognised. Furthermore, in a case the useris in the vicinity of the tourist spot, it is possible to distinguishwhether the user is doing sightseeing on foot, the user is doingshopping, or the user is moving in a train. Accordingly, a method ofinformation presentation such as presentation of tourist spotinformation in a case of sightseeing on foot, presentation ofinformation on a souvenir shop in a case of shopping, and presentationof information on the next tourist spot in a case of moving in a traincan be realised.

Furthermore, even if the user is behind on schedule or moving ahead ofschedule, the behaviour of the user can be grasped from thebehaviour/situation pattern, and thus appropriate information can bepresented by detecting the delay or acceleration. For example, when abehaviour/situation pattern “meal” is actually detected where abehaviour/situation pattern “hot spring” is supposed to be detectedbased on the schedule information as shown in FIG. 32, the delay may benotified to the user. Also, delay in the schedule may be presented tothe user together with the location information on the current locationand the location information described in the schedule information.

As described, by being presented with information in accordance with thesituation at an appropriate timing, a user is enabled to changetransportation means as appropriate based on the presented information,or to change the schedule. Furthermore, a difference to the scheduleinformation can be detected from the behaviour/situation pattern also ina case of the user moving along a route different from that in theschedule information or the user fitting in an event not registered inthe schedule information. Accordingly, it becomes possible to presentinformation (for example, presentation of transportation means) thatappropriately links the behaviour of the user estimated based on thedetected behaviour/situation pattern and the behaviour scheduled next.

Described in the following is a configuration of the behaviour/situationanalysis system 20 that is capable of presenting appropriate informationto a user at an appropriate time by correlating schedule information andthe contents of a detected behaviour/situation pattern with each otheras described above

<2-2: Overall Configuration of System>

First, an overall system configuration of the behaviour/situationanalysis system 20 according to the present embodiment will be describedwith reference to FIG. 33. FIG. 33 is an explanatory diagram showing anexample of the overall system configuration of the behaviour/situationanalysis system 20 according to the present embodiment. Note thatstructural elements that have substantially the same function as thoseof the behaviour/situation analysis system 10 according to the firstembodiment described above are denoted with the same reference numerals,and repeated explanation of these structural elements is omitted.

As shown in FIG. 33, the behaviour/situation analysis system 20 mainlyincludes a motion sensor 102, a location sensor 104, a time/calendarinformation acquisition unit 106, a movement/state recognition unit 108,a geo-categorisation unit 110, and a behaviour/situation recognitionunit 112. Furthermore, the behaviour/situation analysis system 20includes a history storage unit 202, a schedule storage unit 204, abehaviour verification unit 206, a behaviour prediction unit 208, and anapplication display unit 210.

When a user performs a behaviour, first, sensor data is detected by themotion sensor 102. The sensor data detected by the motion sensor 102 isinput to the movement/state recognition unit 108. Furthermore, locationinformation indicating the current location is acquired by the locationsensor 104. Then, the location information on the current locationacquired by the location sensor 104 is input to the geo-categorisationunit 110.

When the sensor data is input, the movement/state recognition unit 108detects a movement/state pattern by using the sensor data. Then, themovement/state pattern detected by the movement/state recognition unit108 is input to the behaviour/situation recognition unit 112. Also, whenthe location information on the current location is input, thegeo-categorisation unit 110 acquires map information MP, and selects ageo category code corresponding to the current location by using theacquired map information MP. Furthermore, the geo-categorisation unit110 calculates a histogram relating to the geo category. The geocategory code selected by the geo-categorisation unit 110 is input tothe behaviour/situation recognition unit 112.

As described above, the movement/state pattern and the geo category codeare input to the behaviour/situation recognition unit 112 respectivelyfrom the movement/state recognition unit 108 and the geo-categorisationunit 110. Also, the sensor data is input to the behaviour/situationrecognition unit 112 via the movement/state recognition unit 108.Furthermore, the location information on the current location is inputto the behaviour/situation recognition unit 112 via thegeo-categorisation unit 110. Furthermore, time/calendar information isinput to the behaviour/situation recognition unit 112 from thetime/calendar information acquisition unit 106.

Thus, the behaviour/situation recognition unit 112 detects abehaviour/situation pattern based on the movement/state pattern, the geocategory code (histogram) and the time/calendar information that havebeen input. Additionally, the behaviour/situation pattern detectionmethod used here may be based on the rule-based determination or on thelearning model determination. The behaviour/situation pattern detectedby the behaviour/situation recognition unit 112 is recorded in thehistory storage unit 202 together with the location information on thecurrent location. Moreover, in the following explanation, pieces ofhistory information on the behaviour/situation patterns accumulated inthe history storage unit 202 may be referred to as a behaviour history.Similarly, pieces of the location information on the current locationaccumulated in the history storage unit 202 may be referred to aslocation history.

The behaviour history and the location history accumulated in thehistory storage unit 202 are read by the behaviour verification unit 206or the behaviour prediction unit 208. The behaviour verification unit206 is means for verifying schedule information against the actualbehaviour/situation pattern. The schedule information is recorded in theschedule storage unit 204. Accordingly, the behaviour verification unit206 compares the contents of the schedule information recorded on theschedule storage unit 204 against the information on the currentlocation detected by the location sensor 104 and the behaviour/situationpattern detected by the behaviour/situation recognition unit 112. In acase the contents of the schedule information match the information ofthe current location and the behaviour/situation pattern, the behaviourverification unit 206 inputs information indicating match to theapplication display unit 210.

On the other hand, in a case the contents of the schedule information donot match the information of the current location and thebehaviour/situation pattern, the behaviour verification unit 206determines whether or not contents matching the information on thecurrent location and the behaviour/situation pattern exist in thecontents from the past or for the future registered in the scheduleinformation. In a case contents matching the information on the currentlocation and the behaviour/situation pattern exist in the contents fromthe past or for the future registered in the schedule information, thebehaviour verification unit 206 inputs, to the application display unit210, information indicating match together with the matching contentsfrom the past or for the future. On the other hand, in a case contentsmatching the information on the current location and thebehaviour/situation pattern do not exist in the contents from the pastor for the future registered in the schedule information, the behaviourverification unit 206 reads the behaviour history and the locationhistory from the history storage unit 202.

Then, the behaviour verification unit 206 compares the contentsregistered in the schedule information against the behaviour history andthe location history that have been read, and detects a time point atwhich the behaviour/situation pattern mismatched the contents ofbehaviour registered in the schedule information. Then, the behaviourverification unit 206 inputs, to the application display unit 210,information indicating the detected time point of occurrence of mismatchand information on the location history and the behaviour history ofthat time point.

Furthermore, the behaviour verification unit 206 repeats the comparisonprocess described above until the location information input from thelocation sensor 104 and the behaviour/situation pattern input from thebehaviour/situation recognition unit 112 match the contents of theschedule information. Then, the behaviour verification unit 206 inputsto the application display unit 210, at the time point of matching,information indicating match and the location information and thebehaviour/situation pattern of that time point.

Furthermore, location information and a result of behaviour/situationpattern prediction are input from the behaviour prediction unit 208 tothe behaviour verification unit 206. For example, in a case there is noschedule information registered in the schedule storage unit 204, thebehaviour verification unit 206 compares the result ofbehaviour/situation pattern prediction input by the behaviour predictionunit 208 and the behaviour/situation pattern actually detected by thebehaviour/situation recognition unit 112. Then, the behaviourverification unit 206 inputs the result of comparison to the applicationdisplay unit 210. The result of prediction by the behaviour predictionunit 208 is used, for example, in a case where information on a place auser is likely to visit after the current location or informationaccording to the behaviour/situation pattern at that place is supposedto be presented but no schedule information is registered.

The behaviour prediction unit 208 reads information on the behaviourhistory and location history accumulated in the history storage unit202, and predicts the behaviour/situation pattern and the locationinformation for the future based on the pieces of information that havebeen read. For example, the behaviour prediction unit 208 uses thebehaviour history and the like read from the history storage unit 202,and predicts the next behaviour/situation pattern of the user based on astochastic location transition model. As the stochastic locationtransition model, a method of estimating a transition probability on thebasis of location clustering described later is used, for example.Furthermore, although not shown in FIG. 33, in a case behaviourhistories and location histories of other people are recorded in thehistory storage unit 202, the behaviour prediction unit 208 may alsoread the behaviour histories and the like of other people and use themfor the prediction of the behaviour/situation pattern. By using thebehaviour histories of other people, prediction of a behaviour in aplace for which there are no behaviour history and the like of the userhimself/herself becomes possible (for example, a behaviour that manypeople are predicted to take is presumed).

As described, the schedule information of the past, present and futureand information on the behaviour/situation pattern or the like that iscurrently detected are verified against each other by the behaviourverification unit 206. Furthermore, the schedule information of thepast, present and future and information on the behaviour/situationpattern that was detected in the past or the like are verified againsteach other by the behaviour verification unit 206. Furthermore,information on a behaviour/situation pattern or the like of the futurepresumed by the behaviour prediction unit 208 and the information on thebehaviour/situation pattern or the like that is currently detected areverified against each other by the behaviour verification unit 206.These verification results are input to the application display unit210. Furthermore, the prediction result by the behaviour prediction unit208 is also input to the application display unit 210.

When these pieces of information are input, the application display unit210 presents to the user appropriate information in accordance with theinput information by using an application. Furthermore, the applicationdisplay unit 210 displays an application used by the user to registerschedule information or to manage the schedule information. Furthermore,the application display unit 210 displays an application for displayinga map, or displays, by the application, a result of verification by thebehaviour verification unit 206 or the like. For example, as shown inFIG. 32, temporal and spatial differences between a scheduled eventregistered in the schedule information and a behaviour/situation patternthat is actually detected are displayed. Additionally, such informationmay be notified by sound.

Heretofore, an overall system configuration of the behaviour/situationanalysis system 20 has been described.

<2-3: Function of Behaviour Prediction Unit 208>

Next, a function of the behaviour prediction unit 208 will be describedwith reference to FIG. 34.

FIG. 34 shows a behaviour prediction method that uses locationclustering. The behaviour prediction unit 208 reads a behaviour historyand a location history from the history storage unit 202, and narrowsdown information on the location history based on the behaviour history.The location history includes location information recorded in variousscenes, such as location information detected during movement of a useron foot, location information detected during transportation by train orbus and location information detected during the user staying still.Therefore, it is extremely difficult to predict the tendency ofbehaviour/situation patterns that the user may take when giving equalimportance to all the pieces of the location information.

Accordingly, the behaviour prediction unit 208 refers to the behaviourhistory, and extracts pieces of location information corresponding tobehaviour/situation patterns “walking” and “still” from the locationhistory. By narrowing down the location information in this manner, theamount of computation relating to behaviour prediction can be reduced.Also, by extracting the behaviour/situation patterns “walking” and“still,” a situation where the user is staying within a certain rangecan be distinguished. Additionally, in a case histories of the length ofstay (or duration), the time/calendar information and the like areincluded in the behaviour history, these histories may be used, and thebehaviour history may be narrowed down to behaviours with long length ofstay (or duration) or the behaviour history may be narrowed down basedon the time of performance of a predicted behaviour. By narrowing downin the manner described above, the accuracy of behaviour prediction canbe improved.

Next, the behaviour prediction unit 208 clusters location information(A) obtained by narrowing down based on the behaviour history. That is,the behaviour prediction unit 208 extracts regions (clusters) wherepoints of the location information concentrate, and groups pointsincluded in each of the extracted regions together with deciding arepresentative point that represents each group. As described, since thebehaviour prediction unit 208 has performed narrowing down based on thebehaviour/situation patterns “walking” and “still,” each cluster is notgreatly expanded. Thus, a cluster map (B) that precisely expresses themain stop points of the user can be created. In the example of FIG. 34,three clusters A, B and C are obtained.

Next, the behaviour prediction unit 208 calculates transitionprobabilities within a cluster and between clusters based on thelocation history. The behaviour history obtained by narrowing down basedon the behaviour/situation patterns “walking” and “still” includesmoving processes (time series location information) of the user.Furthermore, the range of each cluster is specified by the clusteringdescribed above. Accordingly, by using these pieces of information incombination, the behaviour prediction unit 208 can distinguish whichpiece of location information constituting a moving process is includedin which cluster. Also, the behaviour prediction unit 208 candistinguish from which cluster to which cluster the movement accordingto a certain moving process is.

For example, the behaviour prediction unit 208 can detect the number ofmoving processes MAA for movement within the cluster A, the number ofmoving processes MBB for movement within the cluster B, and the numberof moving processes MCC for movement within the cluster C. Also, thebehaviour prediction unit 208 can detect the number of moving processesMAB for movement from the cluster A to the cluster B. Furthermore, thebehaviour prediction unit 208 can predict the number of moving processesMBA for movement from the cluster B to the cluster A. Furthermore, thebehaviour prediction unit 208 can predict the number of moving processesMBC for movement from the cluster B to the cluster C, and the number ofmoving processes MCB for movement from the cluster C to the cluster B.

The transition probabilities within a cluster or between clusters can becalculated based on the ratio between the MAA, MBB, MCC, MAB, MBA, MAC,MCA, MBC and MCB detected in the manner described above. Aftercalculating the transition probabilities in the manner described above,the behaviour prediction unit 208 predicts a behaviour based on thecalculated transition probabilities. For example, transitionprobabilities as shown in (C) of FIG. 34 are calculated (the numberindicates the level of a transition probability), and it is predictedthat, in a case a user is in the cluster A and the behaviour/situationpattern is “shopping,” the user will keep on shopping and then will moveto the cluster B. As described above, even if schedule information isnot registered, the behaviour of a user will be predicted by thebehaviour prediction unit 208 based on the location history and thebehaviour history.

Heretofore, a function of the behaviour prediction unit 208 has beendescribed.

<2-4: Function of Behaviour Verification Unit 206>

Next, an operational flow of the behaviour verification unit 206 will bedescribed with reference to FIGS. 35A to 35C.

(Pre-Processing)

First, reference will be made to FIG. 35A. FIG. 35A is an explanatorydiagram showing a flow of the main processes performed prior to abehaviour verification operation by the behaviour verification unit 206.As shown in FIG. 35A, first, a movement/state pattern is detected basedon sensor data by the movement/state recognition unit 108 (S202). Next,geo category information (geo category code, geo category histogram) isdetected from location information by the geo-categorisation unit 110(S204). Then, the behaviour/situation recognition unit 112 detects abehaviour/situation pattern from the movement/state pattern and the geocategory information (S206).

Next, whether or not schedule information is registered in the schedulestorage unit 204 is determined by the behaviour verification unit 206(S208). In a case schedule information is registered in the schedulestorage unit 204, the behaviour verification unit 206 proceeds to theprocess of step S210. On the other hand, in a case schedule informationis not registered in the schedule storage unit 204, the behaviourverification unit 206 proceeds to the process of step S212. In a case itproceeded to the process of step S210, the behaviour verification unit206 checks the current scheduled event and the next scheduled event fromthe schedule information (S210). On the other hand, in a case itproceeded to the process of step S212, the behaviour verification unit206 acquires information indicating the next behaviour predicted by thebehaviour prediction unit 208 based on the history ofbehaviour/situation patterns (behaviour history/location history)(S212).

(Case where Schedule Information is Registered)

Next, reference will be made to FIG. 35B. In a case schedule informationis registered in the schedule storage unit 204, the behaviourverification unit 206 is aware, by the pre-processing of FIG. 35A, ofthe current scheduled event or the next scheduled event registered inthe schedule information. Accordingly, the behaviour verification unit206 determines whether or not the current location detected by thelocation sensor 104 and the current behaviour/situation pattern detectedby the behaviour/situation recognition unit 112 match the currentscheduled event registered in the schedule information (S222). In a casethey match the current scheduled event, the behaviour verification unit206 proceeds to the process of step S224. On the other hand, in a casethey do not match the current scheduled event, the behaviourverification unit 206 proceeds to the process of step S228.

In a case it proceeded to the process of step S224, the behaviourverification unit 206 determines whether or not the current scheduledevent is already over (S224). In a case the current scheduled event isnot over, the behaviour prediction unit 206 proceeds to the process ofstep S226. On the other hand, in a case the current scheduled event isover, the behaviour verification unit 206 proceeds to the process ofstep S230. In a case it proceeded to the process of step S226, thebehaviour verification unit 206 acquires the current scheduled eventregistered in the schedule information (S226), and proceeds to theprocess of step S240.

In a case the behaviour verification unit 206 proceeded to the processof step S228 based on the determination process of step S222, thebehaviour verification unit 206 determines whether or not the locationinformation and the behaviour/situation pattern currently detected matchthe next scheduled event (S228). In a case they match the next scheduledevent, the behaviour verification unit 206 proceeds to the process ofstep S230. On the other hand, in a case they do not match the nextscheduled event, the behaviour verification unit 206 proceeds to theprocess of step S232. In a case it proceeded to the process of stepS230, the behaviour verification unit 206 acquires the next scheduledevent registered in the schedule information (S230), and proceeds to theprocess of step S240.

In a case the behaviour verification unit 206 proceeded to the processof step S232 based on the determination process of step S228, thebehaviour verification unit 206 determines whether or not the locationinformation and the behaviour/situation pattern currently detected matcha past scheduled event (S232). In a case they match the past scheduledevent, the behaviour verification unit 206 proceeds to the process ofstep S234. On the other hand, in a case they do not match the pastscheduled event, the behaviour verification unit 206 proceeds to theprocess of step S236. In a case it proceeded to the process of stepS234, the behaviour verification unit 206 acquires the past scheduledevent registered in the schedule information (S234), and proceeds to theprocess of step S240.

In a case the behaviour verification unit 206 proceeded to the processof step S236 based on the determination process of step S232, thebehaviour verification unit 206 acquires behaviour prediction data forthe vicinity of the current location from the behaviour prediction unit208 (S236). Then, the behaviour verification unit 206 acquires, from thebehaviour prediction unit 208, behaviour prediction data for a place theuser is likely to visit next (S238), and proceeds to the process of stepS240.

On proceeding to the process of step S240, the behaviour verificationunit 206 checks the type of a scheduled behaviour (S240). Examples ofthe type of the scheduled behaviour includes a gourmet-related behaviour(meal or the like), an entertainment, a vehicle-related behaviour andthe like. Next, the behaviour verification unit 206 acquires informationaccording to the type of the scheduled behaviour (S242). Then, thebehaviour verification unit 206 inputs the acquired information to theapplication display unit 210, and displays the information on anapplication (S244). Then, the behaviour verification unit 206 ends theseries of processes relating to presentation of information.

According to such configuration, even if a behaviour of the user is notfollowing the scheduled event registered in the schedule information,appropriate information in accordance with the behaviour/situationpattern of the time point can be provided. Furthermore, even if thebehaviour of the user is not registered in the schedule information,appropriate information is presented to the user based on a predictedbehaviour/situation pattern.

(Case where Schedule Information is not Registered)

Next, reference will be made to FIG. 35C. In FIG. 35C, behaviourprediction data is created by the behaviour prediction unit 208 in thepre-processing of FIG. 35A in a case schedule information is notregistered in the schedule storage unit 204. Thus, the behaviourverification unit 206 acquires behaviour prediction data for thevicinity of the current location from the behaviour prediction unit 208(S252). Then, the behaviour verification unit 206 acquires behaviourprediction data for a place the user is likely to visit next (S254).

Next, the behaviour verification unit 206 checks the type of a scheduledbehaviour based on the pieces of behaviour prediction data (S256). Next,the behaviour verification unit 206 acquires information according tothe type of the scheduled behaviour (S258). Then, the behaviourverification unit 206 inputs the acquired information to the applicationdisplay unit 210, and displays the information on an application (S260).Then, the behaviour verification unit 206 ends the series of processesrelating to presentation of information.

According to such configuration, even if schedule information is notregistered in the schedule storage unit 204, the behaviour/situationpattern of a user can be predicted, and appropriate information inaccordance with the scene can be presented to the user.

Heretofore, the second embodiment of the present invention has beendescribed. The present embodiment proposed a technology of correlatingschedule information and a behaviour/situation pattern with each otherand providing a user with appropriate information in accordance with thesituation. Also proposed is a technology of predicting, efficiently andaccurately, a behaviour/situation pattern of a user based on a behaviourhistory and a location history. By using these technologies, moreeffective information can be provided to a user according to the scene.

3: Third Embodiment

Next, the third embodiment of the present invention will be described.The present embodiment relates to a method of using abehaviour/situation pattern obtained by the behaviour/situation patterndetection method described in the first embodiment described above.Particularly, the technology of the present embodiment relates to atechnology of finely controlling a notification timing of ToDoinformation that is registered as a future scheduled event of a user.The present embodiment further relates to a technology of sharing theToDo information among multiple users and adequately controllingnotification receivers according to the behaviour/situation pattern ofeach user.

<3-1: Overview of System>

First, an overview of a function to be realised by a behaviour/situationanalysis system 30 according to the present embodiment will be describedwith reference to FIGS. 36 and 37. FIGS. 36 and 37 are explanatorydiagrams showing an effective method of presenting information based oncorrelation with a registration/notification application for ToDoinformation.

First, reference will be made to FIG. 36. FIG. 36 illustrates aregistration screen of an application for registering ToDo informationand a display screen for the registered ToDo information. On theregistration screen, “deadline” input box for setting a time period(present to deadline) for notifying registered ToDo information,“display contents” box for registering ToDo information desired to benotified, a timing box (“type of behaviour”) for setting a notificationtiming, and a sharer selection box (“sharing of ToDo?”) for setting asharer are provided.

A behaviour/situation pattern included in a specific behaviour list canbe selectively input in the timing box of the registration screen. Also,a group included in a specific Group list can be selectively input inthe sharer selection box of the registration screen. The behaviour listincludes behaviour/situation patterns such as “working,” “eating,”shopping,” “household chores” and “viewing,” for example. Also, theGroup list includes groups such as “family,” “club,” “baseball team,”“work related” and “workmate,” for example.

When each of the items described above is registered by using theregistration screen, ToDo information is displayed on the display screenaccording to a result of behaviour/situation pattern detection. Forexample, in a case “on way home” is selected from the behaviour list andis registered, ToDo information is displayed on the display screen at atiming the behaviour/situation pattern detected in accordance with thebehaviour of a user is “on way home.” In the example of FIG. 36, ashopping list of “things to pick up on way home” is registered as theToDo information, and the registered shopping list, “things to pick upon way home” and “go to bookstore,” is displayed at a timing abehaviour/situation pattern of “on way home” is detected.

As described, by having ToDo information displayed in accordance with abehaviour/situation pattern, appropriate ToDo information isautomatically displayed without individually setting the display timingfor each piece of ToDo information. Accordingly, a user can be saved thetrouble of registering, and also, ToDo information can be automaticallydisplayed at an appropriate timing even if a behaviour/situation patternof the user is different from that which was scheduled.

For example, if a display timing of ToDo information is set to a normaltime of getting home, ToDo information is displayed at a differenttiming from the actual time of getting home in a case the time ofgetting home changed. However, with the configuration of FIG. 36, suchfaulty situation will not occur.

Additionally, a behaviour/situation pattern may be automaticallyselected from the behaviour list, at the time display contents of ToDoinformation is input on the registration screen, according to the inputcontents. Additionally, the behaviour/situation pattern to beautomatically selected may be set in advance or may be set by using adetermination model automatically built by machine learning.

For example, as shown in FIG. 37, in a case the display contents of ToDoinformation is “shopping list,” behaviour/situation patterns “moving(train, bus, car, bicycle),” “on way home” and “shopping” correspondingto “shopping list” may be automatically selected. Similarly, “moving (onfoot),” and “walking” may be automatically selected for “message list,”“shopping” and “after shopping” for “household accounts,” “meal” and“after meal” for “calorie check,” “moving (train, bus)” for “Englishlistening,” and “household chores” and “TV viewing” for “recordedprogram viewing.”

As described, with ToDo information being effectively displayed at anappropriate timing in accordance with a behaviour/situation pattern, itbecomes possible to use the ToDo information more effectively. Also,with ToDo information being shared among multiple users as will bedescribed later and a notification receiver of the ToDo informationbeing selected by using the behaviour/situation pattern, inappropriatenotification of the ToDo information to an unrelated user can beavoided, thereby reducing the irritation of the user, and at the sametime, effective provision of the ToDo information can be realised. Inthe following, a configuration of the behaviour/situation analysissystem 30 capable of realising such function will be described indetail.

<3-2: Overall Configuration of System>

First, an overall system configuration of the behaviour/situationanalysis system 30 according to the present embodiment will be describedwith reference to FIG. 38. FIG. 38 is an explanatory diagram showing anexample of the overall system configuration of the behaviour/situationanalysis system 30 according to the present embodiment. Note thatstructural elements that have substantially the same function as thoseof the behaviour/situation analysis system 10 according to the firstembodiment described above are denoted with the same reference numerals,and repeated explanation of these structural elements is omitted.

As shown in FIG. 38, the behaviour/situation analysis system 30 mainlyincludes a motion sensor 102, a location sensor 104, a time/calendarinformation acquisition unit 106, a movement/state recognition unit 108,a geo-categorisation unit 110, and a behaviour/situation recognitionunit 112. Furthermore, the behaviour/situation analysis system 30includes a ToDo registration/notification application 302, a ToDomanagement unit 304, and a database storage unit 306.

When a user performs a behaviour, first, sensor data is detected by themotion sensor 102. The sensor data detected by the motion sensor 102 isinput to the movement/state recognition unit 108. Furthermore, locationinformation indicating the current location is acquired by the locationsensor 104. Then, the location information on the current locationacquired by the location sensor 104 is input to the geo-categorisationunit 110.

When the sensor data is input, the movement/state recognition unit 108detects a movement/state pattern by using the sensor data. Then, themovement/state pattern detected by the movement/state recognition unit108 is input to the behaviour/situation recognition unit 112. Also, whenthe location information on the current location is input, thegeo-categorisation unit 110 acquires map information MP, and selects ageo category code corresponding to the current location by using theacquired map information MP. Furthermore, the geo-categorisation unit110 calculates a histogram relating to the geo category. The geocategory code selected by the geo-categorisation unit 110 is input tothe behaviour/situation recognition unit 112.

As described, the movement/state pattern and the geo category code areinput to the behaviour/situation recognition unit 112 respectively fromthe movement/state recognition unit 108 and the geo-categorisation unit110. Also, the sensor data is input to the behaviour/situationrecognition unit 112 via the movement/state recognition unit 108.Furthermore, the location information on the current location is inputto the behaviour/situation recognition unit 112 via thegeo-categorisation unit 110. Furthermore, time/calendar information isinput to the behaviour/situation recognition unit 112 from thetime/calendar information acquisition unit 106.

Thus, the behaviour/situation recognition unit 112 detects abehaviour/situation pattern based on the movement/state pattern, the geocategory code (histogram) and the time/calendar information that havebeen input. Additionally, the behaviour/situation pattern detectionmethod used here may be based on the rule-based determination or on thelearning model determination. The behaviour/situation pattern detectedby the behaviour/situation recognition unit 112 is input to the ToDoregistration/notification application 302. The ToDoregistration/notification application 302 is means for presenting a userwith ToDo information (contents on a display screen) together withproviding an input interface that is used at the time registration ofthe ToDo information (contents on a registration screen) by the user.

When a deadline, display contents, the behaviour/situation pattern, asharing target and the like (hereinafter, registration information) areregistered in the ToDo registration/notification application 302, theregistration information is input from the ToDoregistration/notification application 302 to the ToDo management unit304. When the registration information is input, the ToDo managementunit 304 stores the registration information in the database storageunit 306. Furthermore, the ToDo registration/notification application302 inputs, to the ToDo management unit 304, the behaviour/situationpattern input from the behaviour/situation recognition unit 112.

When the behaviour/situation pattern is input, the ToDo management unit304 refers to the registration information stored in the databasestorage unit 306, and inputs ToDo information matching the inputbehaviour/situation pattern to the ToDo registration/notificationapplication 302. When the ToDo information is input from the ToDomanagement unit 304, the ToDo registration/notification application 302displays the input ToDo information on a display screen.

Additionally, behaviour/situation patterns of multiple users are inputto the ToDo management unit 304. Accordingly, the ToDo management unit304 selects, based on the registration information stored in thedatabase storage unit 306, a provision target of the ToDo informationwhile taking into consideration the multiple behaviour/situationpatterns that have been input and information on a group the multipleusers belong to. Then, the ToDo management unit 304 inputs the ToDoinformation in the ToDo registration/notification applications 302 ofthe selected provision targets. When the ToDo information is input fromthe ToDo management unit 304, each ToDo registration/notificationapplication 302 displays the input ToDo information on a display screen.

According to such configuration, ToDo information is provided to anappropriate user at an appropriate timing in accordance with abehaviour/situation pattern.

<3-3: Function of ToDo Management Unit 304>

Next, a function of the ToDo management unit 304 will be described withreference to FIGS. 39 to 44.

(Overview)

First, an overview of operations of registration and notification ofToDo information by the ToDo management unit 304 will be described withreference to FIGS. 39 and 40.

As shown in FIG. 39, when registration information is input from theToDo registration/notification application 302, the ToDo management unit304 stores the registration information in the database storage unit306. A ToDo message DB, a user/group management DB and a user'sbehaviour state DB are provided in the database storage unit 306. TheToDo message DB is a database storing ToDo information input as theregistration information. The user/group management DB is a databasestoring information on a user and a group input as the registrationinformation. The user's behaviour state DB is a database storing abehaviour/situation pattern of a user stored in the user/groupmanagement DB. The ToDo management unit 304 stores the registrationinformation in these databases.

As described above, a behaviour/situation pattern is input to the ToDomanagement unit 304 from each user. Accordingly, the ToDo managementunit 304 extracts, from the ToDo message DB, ToDo information matching abehaviour/situation pattern that has been input, and provides the ToDoinformation to a user who has input the behaviour/situation pattern. Atthis time, if a group is linked to the ToDo information, a belonginggroup of a user with a matching behaviour/situation pattern is searchedin the user/group management DB and the ToDo information is providedalso to users belonging to the group linked to the ToDo information.

According to such configuration, a notification timing of ToDoinformation and a notification target of the ToDo information aredecided according to a behaviour/situation pattern, and the ToDoinformation is notified to the notification target at a timing accordingto the behaviour/situation pattern.

As shown in FIG. 40, when the ToDo information is notified and a usercompletes the behaviour of ToDo, the user inputs a ToDo completionnotification to the ToDo management unit 304. When the completionnotification is input, the ToDo management unit 304 deletes the ToDoinformation corresponding to the completion notification from the ToDomessage DB included in the database storage unit 306. Then, the ToDomanagement unit 304 sends a completion notification to users to whom theToDo information has been notified.

With the completion notification sent to each user, each user can knowthe completion of the ToDo. For example, in a case a ToDo that has onlyto be performed by one member of the group is notified, each user of thegroup can know the completion of the ToDo by being notified, by thecompletion notification, of the completion of the ToDo by a user.

The registration, notification, completion registration and completionnotification of a ToDo are performed in the manner described above. Adetailed functional configuration of the ToDo management unit 304 forrealising these functions will be described in the following.

(Detailed Configuration, Operational Flow)

Next, a detailed functional configuration of the ToDo management unit304 and an operational flow of the ToDo management unit 304 will bedescribed with reference to FIG. 41 and FIGS. 42A to 42D. FIG. 41 is anexplanatory diagram showing an example of a detailed functionalconfiguration of the ToDo management unit 304. FIGS. 42A to 42D areexplanatory diagrams showing operational flows of the ToDo managementunit 304.

As shown in FIG. 41, the ToDo management unit 304 is configured from aToDo registration unit 312, a ToDo completion registration unit 314, aToDo synchronisation unit 316, a ToDo notification unit 318, and a timer320.

(Configuration and Operation of Todo Registration Unit 312)

Registration information is input to the ToDo registration unit 312 atthe time of registration. The registration information input at thistime includes information such as an addressee user, a range ofnotification receivers, the type of a trigger behaviour (abehaviour/situation pattern included in a behaviour list), a message(ToDo information) and repetition/non-repetition. When these pieces ofregistration information are input, the ToDo registration unit 312stores these pieces of information in the database storage unit 306.

Specifically, an operation shown in FIG. 42A is performed. As shown inFIG. 42A, when registration information and information (ID) or the likeon a user who has input the registration information are input, the ToDoregistration unit 312 refers to the user/group management DB, anddetermines whether a user corresponding to the input registrationinformation exists or not (S302). In a case a corresponding user existsin the user/group management DB, the ToDo registration unit 312registers a message included in the registration information in the ToDomessage DB (S304). In a case there are multiple corresponding users inthe user/group management DB, the process of step S304 are performed foreach of all the corresponding users.

(Configuration and Operation of Todo Completion Registration Unit 314)

An ID of a completed ToDo is input to the ToDo completion registrationunit 314. As shown in FIG. 42B, when an ID is input, the ToDo completionregistration unit 314 determines whether or not there is ToDoinformation corresponding to the input ID (S312). Then, the ToDocompletion registration unit 314 ends the series of processes in a casecorresponding ToDo information is not registered, and proceeds to theprocess of step S314 in a case corresponding ToDo information isregistered. In a case the ToDo completion registration unit 314proceeded to the process of step S314, the ToDo completion registrationunit 314 sets the status of the corresponding ToDo informationregistered in the ToDo message DB to “complete” (S314). Moreover, in acase of ToDo information with no possibility of being repeatedly used,such ToDo information may be deleted.

(Configuration and Operation of Todo Synchronisation Unit 316)

A notification of ToDo information update performed by the ToDoregistration unit 312 and the ToDo completion registration unit 314 isinput to the ToDo synchronisation unit 316. As shown in FIG. 42C, theToDo synchronisation unit 316 first determines whether or not there is aToDo in relation to which a notification process (registrationnotification, completion notification) is not performed (S322), and endsthe series of processes in a case there is no ToDo that is not notified.On the other hand, in a case there is a ToDo that is not notified, theToDo synchronisation unit 316 selects the ToDo that is not notified(S324), and inputs the result of selection to the ToDo notification unit318.

(Configuration and Operation of Todo Notification Unit 318)

Information indicating a ToDo that is not notified is input to the ToDonotification unit 318 from the ToDo synchronisation unit 316. As shownin FIG. 42D, the ToDo notification unit 318 first determines whether abehaviour/situation pattern of a user registered in the user's behaviourstate DB and a trigger of the ToDo that is not notified match each other(S332), and, in a case they do not match each other, ends the series ofprocesses. On the other hand, in a case they match each other, the ToDonotification unit 318 proceeds to the process of step S334, and notifiesa user of the ToDo that is not notified (S334).

Heretofore, the detailed configuration and operation of the ToDomanagement unit 304 have been described. Additionally, a member list, agroup, a user, a ToDo list, a ToDo item and a behaviour/situationpattern are registered, being linked to each other, in the databasestorage unit 306 in a database format. By using such linked database,ToDo information can be provided to an appropriate provision target atan appropriate timing while taking into consideration the relationshipbetween a behaviour/situation pattern, a group and the like.

For example, by linking and registering a ToDo and a behaviour/situationpattern in advance, a user can be presented, at a time abehaviour/situation pattern that is registered is detected, ToDoinformation linked to the behaviour/situation pattern. Also, ToDoinformation can be provided, for example, at the starting point orending point of a behaviour/situation pattern, by taking time/calendarinformation into consideration. For example, in relation to abehaviour/situation pattern “meal,” a ToDo “take a picture of a meal”may be presented at “meal, start” and a ToDo “calorie check” may bepresented at “meal, end.”

Moreover, a registration screen may be displayed instead of ToDo displayat a timing according to a behaviour/situation pattern. According tothis configuration, ToDo registration can be prompted at an appropriatetiming. Furthermore, a method can also be conceived of detecting thecycle or timing of a ToDo by using a registration history of the userhimself/herself or of another user and optimising, by using the resultof detection, a timing or provision target.

Furthermore, a method can also be conceived of optimising the timing ofnotification of a ToDo or a notification target by using, in addition toa behaviour/situation pattern, location information detected by thelocation sensor 104 or a geo category code (regional characteristics andthe like) detected by the geo-categorisation unit 110. For example, anapplication mode of displaying “list of gifts to be purchased” during“shopping” in a commercial area or a department store may be conceived.

(Selection of Notification Target)

In the explanation above, a configuration of sharing ToDo information bymultiple users has been indicated. Here, an explanation will be givenwith reference to FIGS. 43 and 44 on a method of appropriatelyselecting, according to a behaviour/situation pattern, a notificationtarget of ToDo information with sharing of a ToDo taken as a premise.Additionally, the method which will be described here relates to atechnology that, in a case of notifying a ToDo to other people,automatically selects a notification target by taking into considerationthe situations of the people, or automatically determines, according tothe number of notification targets who are related to the contents of aToDo, whether or not the ToDo is to be notified to the entire group.

(Method of Notifying ToDo to User with Matching Behaviour/SituationPattern)

First, reference will be made to FIG. 43. FIG. 43 schematically shows aconfiguration of automatically selecting a user with matchingbehaviour/situation patterns and notifying a ToDo to the selected user.In the example of FIG. 43, a case is assumed where ToDo information“reservation at a restaurant” is notified to a user whosebehaviour/situation pattern is “moving.” The ToDo information“reservation at a restaurant” is an action that has only to be performedby one member of the group.

Also, it is something that is preferably requested to a user whose isnot in a busy situation. In the example of FIG. 43, of the three usersin the group, two are “working” and one is “moving.” It would beinconsiderate to ask a user whose behaviour/situation pattern is“working” to perform the ToDo of “reservation at a restaurant.” Also, if“reservation at a restaurant” is notified during “working,” the user whohas received the notification may feel irritated by the user who hassent the ToDo. Furthermore, it would be bothersome to confirm thesituation of a user who is far away by telephone or the like and to askhim the “reservation at a restaurant.”

However, when using the technology of the present embodiment, the ToDoinformation “reservation at a restaurant” can be automatically sent onlyto a user whose behaviour/situation pattern is “moving.” To realise suchfunction, first, the ToDo management unit 304 monitors thebehaviour/situation pattern input by each user in a group, and selects auser corresponding to a behaviour/situation pattern “moving.” In a caseone user is selected, the ToDo management unit 304 sends the ToDoinformation to the user. Also, in a case multiple users are selected,the ToDo management unit 304 may send the ToDo information to themultiple users, or may send the ToDo information to one user selectedamong the multiple users.

For example, in a case of selecting one user among the multiple users,the ToDo management unit 304 may refer to location information, a geocategory code (histogram) or the like, in addition to thebehaviour/situation pattern. For example, in a case where there arethree users whose behaviour/situation patterns are “moving” who arerespectively “moving (train),” “moving (car)” and “moving (on foot),” itis desirable that the ToDo information is preferentially notified to theuser who is “moving (on foot).”

Furthermore, in a case there are three users who are respectively in a“shopping area,” a “mountainous area” and a “business district”according to geo category codes (histogram), it is desirable that theToDo information is preferentially notified to the user who is in the“shopping area.” As described, by narrowing down the notificationtargets of ToDo information by using information such as abehaviour/situation pattern, a geo category code and the like, a moreappropriate notification target can be selected.

(Method of Selecting Notification Targets according to Number of Userswith Matching Behaviour/Situation Patterns)

Next, reference will be made to FIG. 44. FIG. 44 schematically shows aconfiguration of automatically determining, according to the number ofusers with matching behaviour/situation patterns, whether or not tonotify all the users in a group of ToDo information. In the example ofFIG. 44, the number of users in the same group with matchingbehaviour/situation patterns is counted, and in a case more than halfthe group members are of the matching behaviour/situation pattern, theToDo information is notified to all the users in the group. In thisexample, a behaviour/situation pattern “moving” and ToDo information“let's go out for dinner” are set.

In a case of making a proposal “let's go out for dinner” to users in agroup, one may wish to know beforehand the number of users who will beaccepting the proposal. Also, one might think to make such proposal ifthe number of users who will accept the proposal is large. However,normally, the number of users who will be accepting a proposal is notknown until a proposal is made. However, depending on the contents of aproposal, it is possible to predict the number of users who will beaccepting the proposal from the behaviour/situation patterns.

As shown in the example of FIG. 44, it is highly possible that a userwho is in a situation “working” will not accept the proposal “let's goout for dinner.” On the other hand, it is highly possible that a userwho is, for example, “moving” will accept the proposal. Therefore, byspecifying a behaviour/situation pattern and counting the number ofusers who match the specified behaviour/situation pattern, the number ofpeople who will be accepting the proposal can be estimated. Accordingly,by determining, based on the result of estimation, whether or not tomake a proposal (send ToDo information), it becomes possible to avoidsending out a fruitless proposal. Moreover, the condition can be changeddepending on the contents of a ToDo; for example, the condition can bethe matching of the behaviour/situation patterns of more than half theusers, the matching of the behaviour/situation pattern of at least oneuser, or the matching of the behaviour/situation patterns of all theusers.

By using such method, a proposal which no one will accept can beprevented from being made, and effective ToDo information can be sent tothe entire group. Although a configuration is shown in the example ofFIG. 44 where ToDo information is sent to the entire group, thetransmission target of ToDo information is not limited to such. Also, anexplanation has been made here taking ToDo information as an example,but the method of FIG. 44 can be applied to any information distributiontechnology. For example, this method can be applied to mail/newsdelivery, software distribution, delivery of video content, delivery ofmusic content, or the like.

(Input Aid)

Next, a method of aiding input on a registration screen will bedescribed. As illustrated in FIG. 36, at the time of registration ofToDo information, an operation of selecting from a behaviour list and anoperation of selecting from a Group list have to be performed, besidesoperations of inputting deadline information and display contents. Asillustrated in FIG. 26, types of behaviour/situation patterns that canbe described in the behaviour list are various. Thus, it is bothersometo search for a desired behaviour/situation pattern in the behaviourlist. Accordingly, the inventors of the present invention have devised amethod of appropriately narrowing down the behaviour list to bedisplayed.

(Narrowing Down of Behaviour List According to Input Contents)

First, a method can be conceived of narrowing down to a behaviour listfrom which a selection is likely to be made, according to the contentsof text input by a user in a display contents section. For example,groups of words related to respective behaviour/situation patterns areprepared, and a score indicating a degree of association is assigned toeach word. The ToDo registration/notification application 302 extracts aword included in the group of words from an input text, and calculates,based on the score of the extracted word, a degree of associationbetween the text and a behaviour/situation pattern. Also, the ToDoregistration/notification application 302 arranges, in order from thehighest, behaviour/situation patterns for which calculated degrees ofassociation are greater than a specific value, and displays thebehaviour/situation patterns as the behaviour list.

By preferentially displaying behaviour/situation patterns with highdegrees of association in this manner, a desired behaviour/situationpattern can be efficiently selected from the behaviour/situation list.Additionally, a method of morphological analysis or the like can be usedfor the method of extracting a word. Also, the score indicating thedegree of association between a behaviour/situation pattern and a wordmay be set in advance or may be calculated by a statistical method.

For example, a method can be conceived of accumulating, as historyinformation, the type of a word used at the time of selection of acertain behaviour/situation pattern and its appearance frequency andsetting a frequently used word as a word with high degree of association(score). Also, a method can be conceived of setting a high degree ofassociation to a word included in the expression of abehaviour/situation pattern. For example, in a case of “moving (train),”high degrees of association are set for “movement” and “train.”Additionally, a score can be weighted by the appearance frequency ofeach word appearing in a text.

(Narrowing Down based on Behaviour/Situation Pattern or the Like)

Next, a method of presenting a candidate for an input character based onthe current behaviour/situation pattern or the like, and a method ofnarrowing down a behaviour list or a sharing list will be introduced.For example, there can be conceived a method of preferentiallypresenting, as an input candidate, a word relating to the latitude andlongitude of the current location, the regional characteristics (geocategory code) or a behaviour/situation pattern (HC behaviour).Similarly, there can be conceived a method of narrowing down candidatesfor the behaviour list or the sharing list based on a word relating tothe latitude and longitude of the current location, the regionalcharacteristics (geo category code) or a behaviour/situation pattern (HCbehaviour).

For example, there is a method of predicting, based on the pastregistration history, a word that is highly probable to be input underthe current situation and presenting the predicted word. Also, there isa method of rearranging the behaviour list so that behaviours are listedin order from a behaviour that is most probable to be registered underthe current situation. Similarly, there is a method of rearranging thesharing list (Group list) so that groups are listed in order from agroup that is most probable to be registered according to the pastregistration history. Note that levels of importance are preferably setin the order of “HC behaviour>regionalcharacteristics>latitude/longitude,” and a probability is preferablycalculated from each element and weighting is preferably performedaccording to the level of importance.

(Correlation Between Behaviour List and Sharing List)

In a case a behaviour/situation pattern is selected from a behaviourlist before the input of contents of a ToDo, a method can be conceivedof presenting a candidate of a word having a high degree of associationwith the selected behaviour/situation pattern, thereby aiding the inputof contents. There is also conceived a method of predicting, based onthe past registration history, a word that is highly probable to beinput when a certain behaviour/situation pattern is selected, andpresenting the word as an input candidate. Furthermore, there is alsoconceived a method of arranging, based on the past registration history,sharing groups that are highly possible to be selected in order in aGroup list. Additionally, besides the registration history, informationon the regional characteristics or the latitude/longitude may be used,or those that are newly registered may be preferentially used.

Heretofore, the third embodiment of the present invention has beendescribed. According to the present embodiment, a technology has beenproposed of correlating a ToDo application and a behaviour/situationpattern with each other, and notifying a ToDo to an appropriateprovision target at an appropriate timing according to the situations ofnotification targets. By using this technology, ToDo information can beprovided to a user at an effective timing.

4: Fourth Embodiment

Next, the fourth embodiment of the present invention will be described.The present embodiment relates to a method of using abehaviour/situation pattern obtained by the behaviour/situation patterndetection method described in the first embodiment described above.Particularly, the technology of the present embodiment relates to atechnology of displaying on a screen, according to a behaviour/situationpattern of a user, an application that is highly possible to be used bythe user, and aiding an operation of application selection by the user.Also, by switching setting information, such as operation settings,according to the behaviour/situation pattern, the user can be saved thetrouble of calling up a setting screen and changing the settings everytime the environment changes.

<4-1: Overview of System>

First, an overview of a function realised by a behaviour/situationanalysis system 40 according to the present embodiment will be describedwith reference to FIG. 45. FIG. 45 is an explanatory diagram showingexamples of an operation of the behaviour/situation analysis system 40and a display configuration realised by the operation.

As shown in FIG. 45, in the behaviour/situation analysis system 40,first, location information and sensor data are acquired (S402). Then, abehaviour/situation pattern is detected based on the locationinformation, the sensor data and the like that have been acquired(S404). Next, applications are rearranged based on the detectedbehaviour/situation pattern (S406). For example, if abehaviour/situation pattern “shopping” is detected, applicationsrelating to shopping are preferentially displayed, as illustrated inFIG. 45. According to such configuration, a user is enabled to instantlyfind an appropriate application that is in accordance with the situationeven at the time of using a client device installed with a large varietyof applications, and the convenience is greatly enhanced.

Additionally, each application is associated with a behaviour/situationpattern in advance. However, whether the application itself uses thebehaviour/situation pattern or not is of no matter. Furthermore, alsowith respect to an application to which a behaviour/situation pattern isnot associated, the application and a behaviour/situation pattern can beassociated based on a relationship in a case a relationship between ause history of a user and a history of behaviour/situation patterns isstatistically calculated. Furthermore, different operation settings maybe set for one application depending on the behaviour/situation pattern.According to such configuration, a user interface in accordance with abehaviour/situation pattern can be provided.

<4-2: Overall Configuration of System>

Next, an overall system configuration of the behaviour/situationanalysis system 40 according to the present embodiment will be describedwith reference to FIG. 46. FIG. 46 is an explanatory diagram showing anexample of an overall system configuration of the behaviour/situationanalysis system 40 according to the present embodiment. Note thatstructural elements that have substantially the same function as thoseof the behaviour/situation analysis system 10 according to the firstembodiment described above are denoted with the same reference numerals,and repeated explanation of these structural elements is omitted.

As shown in FIG. 46, the behaviour/situation analysis system 40 mainlyincludes a motion sensor 102, a location sensor 104, a time/calendarinformation acquisition unit 106, a movement/state recognition unit 108,a geo-categorisation unit 110, and a behaviour/situation recognitionunit 112. Furthermore, the behaviour/situation analysis system 40includes a display control unit 402, and a display unit 404.

When a user performs a behaviour, first, sensor data is detected by themotion sensor 102. The sensor data detected by the motion sensor 102 isinput to the movement/state recognition unit 108. Furthermore, locationinformation indicating the current location is acquired by the locationsensor 104. Then, the location information on the current locationacquired by the location sensor 104 is input to the geo-categorisationunit 110.

When the sensor data is input, the movement/state recognition unit 108detects a movement/state pattern by using the sensor data. Then, themovement/state pattern detected by the movement/state recognition unit108 is input to the behaviour/situation recognition unit 112. Also, whenthe location information on the current location is input, thegeo-categorisation unit 110 acquires map information MP, and selects ageo category code corresponding to the current location by using theacquired map information MP. Furthermore, the geo-categorisation unit110 calculates a histogram relating to the geo category. The geocategory code selected by the geo-categorisation unit 110 is input tothe behaviour/situation recognition unit 112.

As described above, the movement/state pattern and the geo category codeare input to the behaviour/situation recognition unit 112 respectivelyfrom the movement/state recognition unit 108 and the geo-categorisationunit 110. Also, the sensor data is input to the behaviour/situationrecognition unit 112 via the movement/state recognition unit 108.Furthermore, the location information on the current location is inputto the behaviour/situation recognition unit 112 via thegeo-categorisation unit 110. Furthermore, time/calendar information isinput to the behaviour/situation recognition unit 112 from thetime/calendar information acquisition unit 106.

Thus, the behaviour/situation recognition unit 112 detects abehaviour/situation pattern based on the movement/state pattern, the geocategory code (histogram) and the time/calendar information that havebeen input. Additionally, the behaviour/situation pattern detectionmethod used here may be based on the rule-based determination or on thelearning model determination. The behaviour/situation pattern detectedby the behaviour/situation recognition unit 112 is input to the displaycontrol unit 402. Besides the behaviour/situation pattern, the locationinformation detected b the location sensor 104 and the geo category codeselected by the geo-categorisation unit 110 may be input to the displaycontrol unit 402.

When these pieces of information are input, the display control unit 402changes, according to the input information, the arrangement ofapplications displayed on the display unit 404. For example, the displaycontrol unit 402 displays, on the display unit 404, only theapplications associated with behaviour/situation pattern. Furthermore,the display control unit 402 calculates, based on a behaviour history ofthe user and a history of application use, the degree of associationbetween the behaviour/situation pattern and each application, anddisplays, on the display unit 404, only the application for which thedegree of association is greater than a specific value. The degree ofassociation used here can be calculated by using a statistical method.Furthermore, the display control unit 402 changes the operation settingsof an application according to the behaviour/situation pattern.

Heretofore, the fourth embodiment of the present invention has beendescribed. By using the technology of the present embodiment, a desiredapplication can be found easily at the time of using a client deviceinstalled with a large variety of applications, and the convenience of auser is greatly enhanced. Also, with the operation settings of anapplication being automatically reset according to a behaviour/situationpattern, a comfortable operating environment can be obtained without auser performing a special setting operation. Moreover, the technology ofthe present embodiment can be applied to, besides the operation settingsof an application, setting items such as a backlight setting, a volumesetting and a power control setting.

<5: Hardware Configuration>

The functions of the server and the client described above can berealised by using the hardware configuration of the informationprocessing apparatus shown in FIG. 47, for example. That is, thefunction of each of the structural elements is realised by controllingthe hardware shown in FIG. 47 by using a computer program. Additionally,the mode of this hardware is arbitrary, and may be a personal computer,a mobile information terminal such as a mobile phone, a PHS or a PDA, agame machine, or various types of information appliances. Moreover, thePHS is an abbreviation for Personal Handy-phone System. Also, the PDA isan abbreviation for Personal Digital Assistant.

As shown in FIG. 47, this hardware mainly includes a CPU 902, a ROM 904,a RAM 906, a host bus 908, and a bridge 910. Furthermore, this hardwareincludes an external bus 912, an interface 914, an input unit 916, anoutput unit 918, a storage unit 920, a drive 922, a connection port 924,and a communication unit 926. Moreover, the CPU is an abbreviation forCentral Processing Unit. Also, the ROM is an abbreviation for Read OnlyMemory. Furthermore, the RAM is an abbreviation for Random AccessMemory.

The CPU 902 functions as an arithmetic processing unit or a controlunit, for example, and controls entire operation or a part of theoperation of each structural element based on various programs recordedon the ROM 904, the RAM 906, the storage unit 920, or a removalrecording medium 928. The ROM 904 is means for storing, for example, aprogram to be loaded on the CPU 902 or data or the like used in anarithmetic operation. The RAM 906 temporarily or perpetually stores, forexample, a program to be loaded on the CPU 902 or various parameters orthe like arbitrarily changed in execution of the program.

These structural elements are connected to each other by, for example,the host bus 908 capable of performing high-speed data transmission. Forits part, the host bus 908 is connected through the bridge 910 to theexternal bus 912 whose data transmission speed is relatively low, forexample. Furthermore, the input unit 916 is, for example, a mouse, akeyboard, a touch panel, a button, a switch, or a lever. Also, the inputunit 916 may be a remote control that can transmit a control signal byusing an infrared ray or other radio waves.

The output unit 918 is, for example, a display device such as a CRT, anLCD, a PDP or an ELD, an audio output device such as a speaker orheadphones, a printer, a mobile phone, or a facsimile, that can visuallyor auditorily notify a user of acquired information. Moreover, the CRTis an abbreviation for Cathode Ray Tube. The LCD is an abbreviation forLiquid Crystal Display. The PDP is an abbreviation for Plasma DisplayPanel. Also, the ELD is an abbreviation for Electro-LuminescenceDisplay.

The storage unit 920 is a device for storing various data. The storageunit 920 is, for example, a magnetic storage device such as a hard diskdrive (HDD), a semiconductor storage device, an optical storage device,or a magneto-optical storage device. The HDD is an abbreviation for HardDisk Drive.

The drive 922 is a device that reads information recorded on the removalrecording medium 928 such as a magnetic disk, an optical disk, amagneto-optical disk or a semiconductor memory, or writes information inthe removal recording medium 928. The removal recording medium 928 is,for example, a DVD medium, a Blu-ray medium, an HD-DVD medium, varioustypes of semiconductor storage media, or the like. Of course, theremoval recording medium 928 may be, for example, an electronic deviceor an IC card on which a non-contact IC chip is mounted. The IC is anabbreviation for Integrated Circuit.

The connection port 924 is a port such as an USB port, an IEEE 1394port, a SCSI, an RS-232 C port, or a port for connecting an externallyconnected device 930 such as an optical audio terminal. The externallyconnected device 930 is, for example, a printer, a mobile music player,a digital camera, a digital video camera, or an IC recorder. Moreover,the USB is an abbreviation for Universal Serial Bus. Also, the SCSI isan abbreviation for Small Computer System Interface.

The communication unit 926 is a communication device to be connected toa network 932, and is, for example, a communication card for a wired orwireless LAN, Bluetooth (registered trademark), or WUSB, an opticalcommunication router, an ADSL router, or various communication modems.The network 932 connected to the communication unit 926 is configuredfrom a wire-connected or wirelessly connected network, and is theInternet, a home-use LAN, infrared communication, visible lightcommunication, broadcasting, or satellite communication, for example.Moreover, the LAN is an abbreviation for Local Area Network. Also, theWUSB is an abbreviation for Wireless USB. Furthermore, the ADSL is anabbreviation for Asymmetric Digital Subscriber Line.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

The present application contains subject matter related to thatdisclosed in Japanese Priority Patent Application JP 2009-230579 filedin the Japan Patent Office on Oct. 2, 2009, the entire content of whichis hereby incorporated by reference.

1. A behaviour pattern analysis system comprising: a mobile terminalincluding a movement sensor that detects a movement of a user andoutputs movement information, a current location information acquisitionunit that acquires information on a current location, a buildinginformation acquisition unit that acquires information on a buildingexisting at a location indicated by the information acquired by thecurrent location information acquisition unit or information onbuildings existing at the current location and in a vicinity of thecurrent location, a first behaviour pattern detection unit that analysesthe movement information output from the movement sensor, and detects afirst behaviour pattern corresponding to the movement information frommultiple first behaviour patterns obtained by classifying behavioursperformed by the user over a relatively short period of time, and atransmission unit that transmits, to a server, the information on abuilding or buildings acquired by the building information acquisitionunit and the first behaviour pattern detected by the first behaviourpattern detection unit; and a server including a reception unit thatreceives, from the mobile terminal, the information on a building orbuildings and the first behaviour pattern, and a second behaviourpattern detection unit that analyses the information on a building orbuildings and the first behaviour pattern received by the receptionunit, and detects a second behaviour pattern corresponding to theinformation on a building or buildings and the first behaviour patternfrom multiple second behaviour patterns obtained by classifyingbehaviours performed by the user over a relatively long period of time.2. The behaviour pattern analysis system according to claim 1, whereinthe second behaviour pattern detection unit creates, by using a specificmachine learning algorithm, a detection model for detecting the secondbehaviour pattern from the information on a building or buildings andthe first behaviour pattern, and detects, by using the created detectionmodel, the second behaviour pattern corresponding to the information ona building or buildings and the first behaviour pattern received by thereception unit.
 3. The behaviour pattern analysis system according toclaim 1, wherein the mobile terminal further includes a time informationacquisition unit that acquires information on a time of a time point ofacquisition of the information on a current location by the currentlocation information acquisition unit, wherein the transmission unittransmits, to the server, the information on a building or buildingsacquired by the building information acquisition unit, the firstbehaviour pattern detected by the first behaviour pattern detection unitand the information on a time acquired by the time informationacquisition unit, wherein the server holds, for each combination of thefirst behaviour pattern and the information on a time, a score mapassigning a score to each combination of the information on a buildingor buildings and the second behaviour pattern, wherein, in a case thescore map is selected based on the first behaviour pattern detected bythe first behaviour pattern detection unit and the information on a timeacquired by the time information acquisition unit, a combination ofscores corresponding to the information on a building existing at thecurrent location acquired by the building information acquisition unitis extracted from the selected score map and a highest score in theextracted combination of scores is a specific value or less, the secondbehaviour pattern detection unit creates, by using a specific machinelearning algorithm, a detection model for detecting the second behaviourpattern from the information on a building or buildings and the firstbehaviour pattern, and detects, by using the created detection model,the second behaviour pattern corresponding to the information on abuilding or buildings and the first behaviour pattern received by thereception unit.
 4. The behaviour pattern analysis system according toclaim 1, wherein the mobile terminal further includes a storage unit inwhich schedule information recording, in a time-series manner, abehaviour of a user capable of being expressed by a combination of thesecond behaviour patterns is stored, a matching determination unit thatreads the schedule information stored in the storage unit, anddetermines whether a present behaviour, a past behaviour and a futurebehaviour of the user recorded in the schedule information and thesecond behaviour pattern detected by the second behaviour patterndetection unit match or not, and a display unit that displays, accordingto a result of determination by the matching determination unit, whetheran actual behaviour matches a schedule recorded in the scheduleinformation, is behind the schedule, or is ahead of the schedule.
 5. Thebehaviour pattern analysis system according to claim 4, wherein theserver further includes a behaviour prediction unit that predicts, byusing a history of the second behaviour patterns detected by the secondbehaviour pattern detection unit, a second behaviour pattern to beperformed by the user next, wherein, in a case of determining that abehaviour of the user matching the second behaviour pattern is notrecorded in the schedule information, the matching determination unitacquires, from the server, the second behaviour pattern predicted by thebehaviour prediction unit and extracts, from the schedule information, abehaviour of the user matching the acquired second behaviour pattern,and wherein the display unit displays information relating to thebehaviour of the user extracted by the matching determination unit. 6.The behaviour pattern analysis system according to claim 5, wherein theserver includes a location information accumulation unit that receives,by the reception unit, the information on a current location acquired bythe current location information acquisition unit and the firstbehaviour pattern detected by the first behaviour pattern detectionunit, and stores the information on a current location and a history ofthe first behaviour patterns in the storage unit, and a clustering unitthat clusters places where the user stays for a long time, by using theinformation on a current location and the history of the first behaviourpatterns accumulated in the storage unit by the location informationaccumulation unit, and calculates a staying probability of staying ateach of the places and a movement probability of moving between theplaces, and wherein the behaviour prediction unit predicts the secondbehaviour pattern to be performed by the user next, based on the stayingprobability and the movement probability calculated by the clusteringunit.
 7. The behaviour pattern analysis system according to claim 1,wherein the behaviour pattern analysis system includes multiple mobileterminals, and wherein the server further includes a notificationinformation storage unit that stores, in association with each other,notification information to be notified at a specific time and aspecific second behaviour pattern, and an information notification unitthat, at the specific time, refers to the second behaviour patterndetected by the second behaviour pattern detection unit based on theinformation on a building or buildings and the first behaviour patternreceived by the reception unit from each of the mobile terminals, andnotifies a mobile terminal corresponding to a second behaviour patternsame as the specific second behaviour pattern of the notificationinformation.
 8. The behaviour pattern analysis system according to claim7, wherein the information notification unit counts the number of mobileterminals corresponding to a second behaviour pattern same as thespecific second behaviour pattern, and in a case the number of themobile terminals is a specific number or more, notifies all of themultiple mobile terminals of the notification information.
 9. A mobileterminal comprising: a movement sensor that detects a movement of a userand outputs movement information; a current location informationacquisition unit that acquires information on a current location; abuilding information acquisition unit that acquires information on abuilding existing at a location indicated by the information acquired bythe current location information acquisition unit or information onbuildings existing at the current location and in a vicinity of thecurrent location; a first behaviour pattern detection unit that analysesthe movement information output from the movement sensor, and detects afirst behaviour pattern corresponding to the movement information frommultiple first behaviour patterns obtained by classifying behavioursperformed by the user over a relatively short period of time; and asecond behaviour pattern detection unit that analyses the information ona building or buildings acquired by the building information acquisitionunit and the first behaviour pattern detected by the first behaviourpattern detection unit, and detects a second behaviour patterncorresponding to the information on a building or buildings and thefirst behaviour pattern from multiple second behaviour patterns obtainedby classifying behaviours performed by the user over a relatively longperiod of time.
 10. The mobile terminal according to claim 9, furthercomprising: a time information acquisition unit that acquiresinformation on a time of a time point of acquisition of the informationof a current location by the current location information acquisitionunit, wherein a score map assigning a score to each combination of theinformation on a building or buildings and the second behaviour patternis provided for each combination of the first behaviour pattern and theinformation on a time, and wherein the second behaviour patterndetection unit selects the score map based on the first behaviourpattern detected by the first behaviour pattern detection unit and theinformation on a time acquired by the time information acquisition unit,extracts, from the selected score map, a combination of scorescorresponding to the information on a building existing at the currentlocation acquired by the building information acquisition unit, anddetects the second behaviour pattern corresponding to a highest score inthe extracted combination of scores.
 11. The mobile terminal accordingto claim 9, further comprising: a time information acquisition unit thatacquires information on a time of a time point of acquisition of theinformation of a current location by the current location informationacquisition unit, wherein a score map assigning a score to eachcombination of the information on a building or buildings and the secondbehaviour pattern is provided for each combination of the firstbehaviour pattern and the information on a time, wherein the buildinginformation acquisition unit acquires, as the information on buildingsexisting at the current location and in a vicinity of the currentlocation, category types of the buildings and the number of buildingsfor each category type, and wherein the second behaviour patterndetection unit selects the score map based on the first behaviourpattern detected by the first behaviour pattern detection unit and theinformation on a time acquired by the time information acquisition unit,extracts, from the selected score map, combinations of scorescorresponding to respective category types acquired by the buildinginformation acquisition unit, normalises, by respective highest scores,each score included in the combinations of scores corresponding torespective category types, performs weighting on the normalisedcombinations of scores corresponding to respective category typesaccording to the number of buildings for each category type acquired bythe building information acquisition unit, and adds, for each secondbehaviour pattern, the weighted scores corresponding to the respectivecategory types, and detects the second behaviour pattern for which aresult of addition is greatest.
 12. The mobile terminal according toclaim 9, further comprising: a display unit on which a display objectfor starting an application associated with the second behaviour patternis displayed; and a display control unit that makes the display unitpreferentially display, according to the second behaviour patterndetected by the second behaviour pattern detection unit, the displayobject associated with the second behaviour pattern.
 13. A behaviourpattern analysis server comprising: a reception unit that receives, froma mobile terminal including a movement sensor that detects a movement ofa user and outputs movement information and a current locationinformation acquisition unit that acquires information on a currentlocation, the movement information and the information on a currentlocation; a building information acquisition unit that acquiresinformation on a building existing at a location indicated by theinformation on a current location received by the reception unit orinformation on buildings existing at the current location and in avicinity of the current location; a first behaviour pattern detectionunit that analyses the movement information received by the receptionunit, and detects a first behaviour pattern corresponding to themovement information from multiple first behaviour patterns obtained byclassifying behaviours performed by the user over a relatively shortperiod of time; and a second behaviour pattern detection unit thatanalyses the information on a building or buildings acquired by thebuilding information acquisition unit and the first behaviour patterndetected by the first behaviour pattern detection unit, and detects asecond behaviour pattern corresponding to the information on a buildingor buildings and the first behaviour pattern from multiple secondbehaviour patterns obtained by classifying behaviours performed by theuser over a relatively long period of time.
 14. A behaviour patternanalysis method comprising the steps of: acquiring movement informationindicating a result of detection by a movement sensor for detecting amovement of a user; acquiring information on a current location;acquiring information on a building existing at a location indicated bythe information on a current location acquired in the step of acquiringinformation on a current location or information on buildings existingat the current location and in a vicinity of the current location;analysing the movement information acquired in the step of acquiringmovement information, and detecting a first behaviour patterncorresponding to the movement information from multiple first behaviourpatterns obtained by classifying behaviours performed by the user over arelatively short period of time; and analysing the information on abuilding or buildings acquired in the step of acquiring information on abuilding or buildings and the first behaviour pattern detected in thestep of analysing the movement information and detecting a firstbehaviour pattern, and detecting a second behaviour patterncorresponding to the information on a building or buildings and thefirst behaviour pattern from multiple second behaviour patterns obtainedby classifying behaviours performed by the user over a relatively longperiod of time.
 15. A program for causing a computer to realise: amovement information acquisition function of acquiring movementinformation indicating a result of detection by a movement sensor fordetecting a movement of a user; a current location informationacquisition function of acquiring information on a current location; abuilding information acquisition function of acquiring information on abuilding existing at the current location indicated by the informationacquired by the current location information acquisition function orinformation on buildings existing at the current location and in avicinity of the current location; a first behaviour pattern detectionfunction of analysing the movement information acquired by the movementinformation acquisition function, and detecting a first behaviourpattern corresponding to the movement information from multiple firstbehaviour patterns obtained by classifying behaviours performed by theuser over a relatively short period of time; and a second behaviourpattern detection function of analysing the information on a building orbuildings acquired by the building information acquisition function andthe first behaviour pattern detected by the first behaviour patterndetection function, and detecting a second behaviour patterncorresponding to the information on a building or buildings and thefirst behaviour pattern from multiple second behaviour patterns obtainedby classifying behaviours performed by the user over a relatively longperiod of time.